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W«  consider  the  economic  behavior  of  a  queueing  system, 
operating  under  a  specified  linear  cost  structure, 
in  which  the  server  may  be  turned  on  and  off.  Optimal 
policies  for  turning  the  server  on  and  off  are  derived 
for  differing  assumptions  about  discounting  of  future 
costs,  length  of  the  planning  horizon,  the  form  of 
the  arrival  stream,  and  the  number  of  servers. 

The  costs  imposed  are:  a  server  start-up  cost,  a 
server  shut-down  cost,  a  cost  per  unit  time  when  the 
server  is  turned  off,  a  cost  per  unit  time  when  the 
server  is  turned  on,  and  a  holding  cost  per  unit  time 
spent  in  the  system  for  each  customer.  We  prove  that 
for  the  single  server  queue  there  is  a  stationary 
optimal  policy  of  the  form:  Turn  the  server  on  when 
n  customers  are  present,  and  turn  it  off  when  the 
system  is  empty. 

For  the  undiscounted,  infinite  horizon  problem  with 
Poisson  arrivals,  an  exact  expression  for  the  cost 
rate  as  a  function  of  n  and  a  closed  form  expression 
for  the  optimal  value  of  n  is  derived;  bounds  are 
obtained  for  the  cost  rate  and  optimal  policy  when 
the  inter-arrival  time  distribution  is  allowed  to  be 
any  member  of  the  class  of  IFR  distributions.  When 
future  costs  are  discounted,  we  obtain  an  equation  for 
the  expected  discounted  cost  as  a  function  of  n  and 
the  interest  rate,  and  prove  that  for  small  interest 
rates  the  optimal  discounted  policy  is  approximately 
the  optimal  undiscounted  policy.  The  recursion  relationship 
to  find  the  optimal  (ncnstat ionary)  policy  for  finite 
horizons  is  developed,  concluding  our  results  for  single 
server  systems,  each  channel  is  restricted  to  have  an 
exponential  service  time  distribution  (possibly  with 
different  rate),  and  the  arrivals  form  a  Poisson 
process.  One  server  is  always  turned  on  and  the  other, 
the  spare  machine,  can  be  turned  on  and  off  at  arbitrary 

times;  we  show  that  the  stationary  optimal  policy  for 
undiscounted  costs  has  the  form:  Turn  the  spare 
machine  on  when  n  customers  are  present,  and  turn  it 
off  when  m  customers  are  in  the  system,  with 
m  <  1  .  We  then  derive  equations  for  finding  the 
optimal  values  of  m  and  n  ;  queue  disciplines  where 
customers  may  be  switched  from  one  server  to  the  other 
are  also  considered. 
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Chapter  I 

INTRODUCTION  AND  PRELIMINARY  RESULT?# 

The  similarity  between  queueing  and  inventory  models  has  long  been 
recognized;  inventory  analysis  generally  includes  an  explicit  cost  struc¬ 
ture  and  a  solution  for  optimal  policies,  but  researchers  in  queueing  theory 
have  been  more  interested  in  the  underlying  probabilistic  structure.  Our 
research  is  directed  towards  finding  optimal  operating  policies  for  a  queue¬ 
ing  system  with  a  linear  cost  structure,  with  emphasis  on  models  with 
Poisson  arrivals. 

1.1  Definitions 

Two  generic  terms  will  be  used  throughout,  server  and  customer.  A 
server  is  a  mechanism  that  performs  an  operation  on  units  fed  into  It; 
these  units  are  referred  to  as  customers .  Thus,  a  server  can  represent  a 
production  line  and  the  customers  can  represent  orders  for  the  product, 
or  the  customer  could  be  people  arriving  at’  a  ticket  window,  and  the  server 
the  ticket  vendor.  When  the  customer  is  being  processed  l>y  the  server, 
he  is  said  to  be  in  service,  and  the  time  he  spends  in  service  is  called 
his  service  time.  While  one  customer  is  in  service,  other  waiting  cus¬ 
tomers  are  in  queue .  that  is,  they  are  present  but  have  not  yet  been  served; 
the  length  of  time  a  customer  waits  in  queue  is  called  his  queueing  or 
wa iting  time,  and  the  queueing  time  plus  the  service  time  of  a  customer 
is  called  his  life  time.  The  system  is  the  queue  and  the  server,  so  the 
number  of  customers  in  the  system  is  the  number  of  customers  in  queue  plus 
the  number  of  customers  in  service. 

The  server  may  not  be  allowed  to  serve  arriving  customers,  i.e.,  it 
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may  be  turned  off.  Time  intervals  when  the  server  is  turned  off  will  be 
called  dormant  periods  and  during  these  periods  the  servers  is  said  to  be 
dormant.  When  the  server  is  turned  on  it  is  running,  and  time  intervals 
when  the  server  is  running  are  called  running  per  iods .  Intervals  when 
customers  are  not  being  served  are  idle  per ? ods  .  they  occur  when  no  customers 
are  present  and/or  when  the  server  is  dormant;  busy  periods  are  time  inter¬ 
vals  when  customers  are  being  served.  This  definition  of  busy  period  cor¬ 
responds  to  the  usual  queueing  terminology,  but  our  definition  of  idle  period 
includes  the  additional  time  when  customers  are  present,  and  the  server  is 
dormant.  A  busy  cycle  is  a  consecutive  busy  and  idle  pt'i  iod. 

Number  of 
customers  in 

the  system 


J - >  t  i  me 

*4 

Figure  1.  A  Typical  Time  Interval 

These  definitions  are  easily  understood  by  referring  to  Figure  1.  The 
interval  [tj.t^l  is  a  dormant  period,  (t^,t^)  is  a  running  period,  (tpt^T 
is  an  idle  period,  (t^.t^l  is  a  busy  period,  and  during  ( t ^ , t^)  the  server 
may  be  running  or  dormant. 

The  economics  of  system  operation  is  influenced  by  the  various  costs 
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involved.  When  the  server  is  dormant,  costs  for  power,  heat,  maintenance, 
etc.  may  still  be  incurred  on  a  per  unit  time  basis;  these  are  called 
dormant  costs.  Activating  and  deactivating  the  server  may  involve  power 
surges,  equipment,  or  manpower  charges;  the  associated  costs  are  called 
start-up  and  shut-down  costs  respective ly,  and  fixed  cos  ts  col  lective ly. 
When  the  server  is  running,  attendant,  fuel,  and  other  costs  may  be  charged 
in  addition  to  the  dormant  cost;  the  sum  of  these  costs  is  the  runn inq 
cost .  These  four  costs  are  the  operating  costs  of  the  server,  but  they 
may  not  represent  the  entire  cost  picture  of  an  operation;  for  example  in  a 
service  such  as  aircraft  repair,  the  airplanes  are  not  productive  during 
their  stay  in  the  repair  depot,  and  this  lost  time  represents  a  cost  to 
the  owner.  Thus,  we  must  consider  a  penalty  for  delaying  the  customer  in 
the  system,  this  penalty  is  called  a  holding  cost. 

1,2.  Assumptions  and  Nota t i on 

Assumptions  for  a  single  server  model  are: 

•  )  The  arrival  stream  -  customers  arrive  singly  in  a  renewal  process;  the 
times  between  successive  arrivals  [cr{ ,  1*1,  2,  ...}  have  distribution 
function  (d .  f . )  A(t) ,  t  >0.  For  most  of  our  results  we  will  require 
A(t)  =  l-e‘Xt. 

b)  The  service  mechanism  -  customers  are  processed  individually  in  their 
order  of  arrival;  the  service  times  [oj ,  i=l,  2,  ...}  are  independent, 
identically  distributed,  non-negative  random  variables  with  d.f . 

B(t)  ,  t  >  0. 

c)  The  cos t  structure  - 

i)  The  dormant  cost  rate  is  Tj  [$/hr.l; 
i  i)  the  running  cost  rate  is  r ^  [$/hr  .1,  >  r  j  ; 
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lii)  the  start-up  cost  is  Rj[$]; 

Iv)  the  shut-down  cost  Is  R^l; 
v)  the  holding  cost  is  h[$ /customer -hr .1. 

All  cost  coefficients  are  assumed  non-negative  and  finite,  and  to 
avoid  trioiality  h  is  positive.  Future  costs  may,  or  may  not  be, 
discounted. 

d)  The  dec i s  i on  process  -  The  server  may  be  turned  on  (or  left  on)  at  any 
point  during  an  idle  period;  and  it  may  be  turned  off  at  service  com¬ 
pletion  epochs  during  a  busy  period,  (note  that  this  precludes  deacti¬ 
vating  the  server  during  the  service  time  of  a  customer). 

Every  possible  policy  of  turning  the  server  on  off  during  the  operating 
horizon  for  the  the  queueing  system  leads  to  a  different  operating  cost. 

The  central  problem  of  this  work  is  to  find  the  optimal  policy  -  i  ,e . ,  the 
pol  icy  which  wi  1 1  minimize  the  tota  1  cost  of  the  operating  horizon. 

Throughout  the  paper  Laplace-Stie 1  tjes  transforms  (LST'S)  of  distri- 

r°°  t 

button  functions  will  be  denoted  by  a  tilde,  e.g.,  ^(s)  =  e  aA(t); 

J  0 

moments  of  d.f.'s  will  be  denoted  by  a  v  with  a  first  subscript  indicating 
the  moment  and  a  second  subscript  indicating  the  d.f.,  e.g.,  the  first 
moment  of  A(t)  is  and  the  second  moment  of  B(t)  is  V2g.  The  quantity 
p  «  Xv^  is  the  system  utilization  factor,  always  assumed  to  be  less  than 
un  i  ty . 

1.3  Previous  Results  Used 

When  the  arrivals  are  a  Poisson  process  and  the  server  is  always  run¬ 
ning,  the  system  forms  an  M/G/l  queue,  a  stochastic  process  that  has  been 
studied  extensively  (see,  for  example,  reference  4).  We  will  often  refer 
to  the  following  results: 
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At  equi  librium,  the  expected  number  of  customers  in  the  system  is 
given  by  the  Pol  laczek-Khi nchine  equation 


(i) 


p2(cb  +  i) 

1  =  p  +  ' 


where  C 


8 
The 


is  the  coefficient  of  variation  of  B(t). 
length  of  a  bcsy  period,  y,  has  d.f.  G(t),  and 


(2)  £(0  =  fits  >  X  -  XG(s)]. 


The  number  of  customers  served  during  a  busy  period,  “H,  and 

00  k 

?(Z)  -  E  ZKp(^k)  satisfies 
k=0 

(3)  F(Z)  =  Z'Stx  -  Xf(Z)]. 


<1 

The  probabi  1 1 ty  that  the  server  is  busy  at  an  arbitrary  point  in  time 
is 


<*>  PB  -  P 

1.4  The  H  /G/1  ttutue 

In  the  next  c!  apter  it  w  i  1 5  be  shown  that  with  Poisson  arrivals  and 
the  assumptions  of  Section  1.2  th-sre  is  an  optimal  operating  policy  of  the 
form; 

(0)  Turn  the  server  on  when  n  customers  are  present, 

and  then  turn  it  off  when  the  system  is  empty. 
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The  queueing  process  formed  by  this  arrival  and  service  pattern  will  be 

called  an  H  /G/l  queue  (when  the  subscript  n  is  set  equal  to  one,  we  have 
n 

the  ordinary  M/G/1  queue).  The  basic  properties  of  this  type  of  queueing 
system  are  given  by  the  following  theorems: 

Theorem  1:  The  idle  period  £,  in  a  M  /G/l  queue  has  d.f.  A  (t)  and 

Aw-tetf- 

Proof:  An  idle  period  will  be  formed  by  n  >  1  independent,  identically 
distributed  inter-arrival  times  so^n(s)  =  [^(s)]n  “  [j+x*!  *  QED. 

Theorem  2;  The  busy  period  y,  in  an  M^/G/l  queue  has  d.f.  G^(t)  and 
1„<0  '[^(s)]n.  "  >  I,  TT0(s)  -^(s). 

Proof:  Let  Tn  be  the  time  to  serve  the  n  customer  present  when  the  busy 

period  begins,  and  k  be  the  nuntoer  of  customers  that  arrive  during 

JL 

Tn<  P0"n  <  t)  =  [B(t)l  =  Bn(t)  because  the  service  times  are 
Independent  and  identically  distributed  random  variables,  and 

E(zk)  =  f  e-*0-*)tdB  (t)  =  [-rtx-xz)!". 

J0  n 

6n(t|k,Tn)  =  T*[G(t)fk 
?n(s|k,Tn)  =  e  STn[?(s)]* 

'6n(5lTn)  =  e*P  t-T„[.  + 

?n(s)  =  {Sts  +  k-X?(s)13"  =  [G(s)3n.  QED. 


I 

i 


nv 

An  immediate  corollary  is  the  vlr  =  — —  since 

IG  l-p 
n 


V1G  " 


nv 


nv 


IB 


1G  l-p  ‘ 


Theorem  3 :  The  number  of  customers  served  in  a  busy  period  of  an  H^/G/l 
queue  has  probability  generating  function  T  (z)  =  [?"(z)]n, 


and  v 


IF 


=  nVjp ,  n  >  1 . 


Proof :  This  theorem  can  be  proved  in  a  manner  analagous  to  Theorem  2;  we 
present  a  more  intuitive  proof  to  help  in  understanding  our  latter 
results . 

Instead  of  serving  the  original  n  customers  first,  serve  any 
one  of  them  and  then  all  the  customers  that  arrive  during  his 
service,  then  those  that  arrive  during  these  services,  etc.  until 
only  the  n-1  original  customers  remain  in  the  queue.  The  number 

i 

served  up  until  this  point  is  the  number  that  would  be  served  during 

* 

the  busy  period  of  an  M/G/l  queue,  which  has  p.g.f.  F (z) .  Pro¬ 
ceeding  in  the  same  manner  with  the  remaining  n-1  customers,  they 
each  generate  an  M/G/l  busy  period,  and  each  busy  period  is  inde- 

A  A 

pendent  and  identically  distributed.  Thus,  F^  (z)  =  [F(z)"ln  and 
V1F  =  nv|F  ’  WD* 

Theorem  4 ;  The  asymptotic  probability  that  the  server  in  an  M^/G/l  queue 
is  idle  is  Pj  =  l-p. 

Proof :  Since  the  arrivals  are  Poisson,  the  sequence  of  idle  and  busy  periods 
forms  an  alternating  renewal  process,  and  the  asymptotic  probability 
of  being  in  an  idle  period  is  P^  =  Therefore,  for 

n  >  I 


"<r> 


1 


,1  .  IB, 

"(x  +  T^> 


1  + 

1-P 


=  1-0 


and  the  cases  n=0  and  n=l  are  the  same,  QED. 

The  importance  of  this  theorem  is  that  for  all  policies  of  the  form 

(0) ,  the  fraction  of  time  the  server  is  busy  is  the  same. 

The  next  four  chapters  are  devoted  to  finding  optimal  operating  policies 

for  an  H^/G/l  queueing  system  with  different  assumptions  about  discounting 

of  total  costs  and  the  length  of  the  planning  horizon.  In  Chapter  III, 

some  bounds  for  a  G  /G/l  queue  will  be  obtained.  The  last  chapter  contains 

n 

a  two-channel  model  in  which  the  service  time  distributions  are  restricted 


to  be  exponent ia  1 . 
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Chapter  II 

THE  EXISTENCE  OF  STATIONARY  OPTIMAL  POLICIES 
FOR  INFINITE  HORIZON  PROBLEMS 

In  Chapters  III  and  IV  we  will  find  stationary  policies  that  minimize 
certain  long-term  objective  functions;  the  purpose  of  this  chapter  is  to 
show  that  there  are  no  non-s tat i ona ry  policies  that  give  a  lower  value  of 
the  objective  function.  The  method  of  proof  will  be  by  formulating  the 
decision  process  as  a  dynamic  programming  problem,  and  proving  that  this 
problem  has  a  stationary  optimal  solution. 

2.1  Dynamic  Programming  Formulation 

Let^  S  be  the  set  of  states  of  a  process  and  A  be  the  set  of  acts 
available;  when  the  process  is  in  state  s  e  S  and  the  act  CL£  A  is  chosen, 

j 

the  process  moves  to  a  new  state  s'  e  S,  where  s'  is  chosen  according  to 
some  probability  distribution  depending  on  s  and  CL  ,  and  gives  a  transition 
reward  r  (s  ,  CL  ,  s').  A  policy  nrr  specifies  which  act  to  choose,  at  a  1 1 
decision  points,  as  a  function  of  the  history  of  the  process.  A  stationary 
policy  can  be  represented  as  a  function,  f,  from  S  into  A  such  that  whenever 
the  process  is  in  state  s,  the  act  d.  =  f(s)  is  chosen.  Thus,  a  stationary 
policy  is  independent  of  the  history  of  the  process,  except  as  summarized  in 
its  current  state. 

The  total  income  from  a  policy  is  the  sum  of  the  transition  rewards 
when  that  policy  is  used;  when  the  transition  rewards  are  random  variables , 
the  expected  income  is  the  sum  of  the  expected  transition  rewards.  If 

This  description  of  dynamic  programming  is  due  to  Blackwell,  reference  2. 
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^(n,t)  denotes  the  expected  income  earned  by  policy  tt  at  time  t,  and  con¬ 
tinuous  discounting  with  Interest  rate  B  is  employed,  then 

r®  _et 

I  (tt , 8 )  =  e  dxi^(TT,t)  is  the  expected  discounted  income  over  an  infinite 
0  T 

A  |  f 

horizon  under  policy  tt.  The  gain  rate  of  policy  rr  is  C  (tt)  =  l  im  =r  dc#(n,  t) . 

T-w  T  u 

When  all  the  rewards  represent  costs,  the  terms  total  cost  and  asymptotic 
cost  rate  may  be  substituted  for  income  and  gain  rate  respectively. 

This  formulation  can  be  applied  to  the  problems  considered  in  this 
thesis  in  the  following  manner. 

Let  A=(l,2)  where  action  1  is  "turn  the  server  off"  and  action  2  is 

"turn  the  server  on";  the  state  of  the  system  is  the  pair  (n,k)  where  n  is 

the  number  of  customers  in  the  system  and  k  indicates  if  the  server  is  run¬ 
ning  or  dormant  and  can  be  written  as  the  denumerable  set 

S  =  {0,0' , 1 , 1 1 ,2 ,2 1 ,  ...}  where  the  prime  indicates  the  server  is  running. 

The  probability  law  that  selects  the  next  state  and  transition  time  is: 
s'  =  s  +  1  if  action  1  is  used,  the  transition  time  is  a  with  d.f.  1-e 
if  action  2  is  used,  s'  will  equal  s-1  +  6  (s )  +  Y,  where  Y  is  the  number 
of  customers  that  arrive  in  a  service  interval,  6(.)  is  the  unit  pulse  at 
the  origin,  the  transition  time  is  a,  which  has  d.f.  B ( t ) .  The  cost  of 

each  transition  is  the  sum  of  the  operating  costs  for  the  server  and  the 

holding  costs  of  the  customers  present  during  the  transition. 

Notice  that  if  we  allcwed  the  instants  when  customers  arrive  during  a 

busy  period  to  be  decision  points,  a  probability  distribution  for  s' 

depending  only  on  s  and  would  not  be  obtained  unless  B(t)  =  1-e”^1*,  and 
the  decision  points  during  an  idle  period  must  be  restricted  to  arrival 
epochs  when  the  arrival  stream  is  not  a  Poisson  process. 


2.2  Problems  with  Discounted  Rewards 


When  an  interest  rate  3  >  0  is  used  to  discount  future  rewards,  we  have 


Theorem  1 


r  21 

(Blackwell)1  :  If  the  state  space  S  is  a  Borel  subset  of  some 
complete  separable  metric  space,  the  action  space  A  is  finite, 
and  the  reward  function  r(*)  is  a  Ba  i  re  function  on  SxAxS ,  then 
there  is  a  stationary  optimal  policy.  That  is,  if  I(tt,s,P) 
is  the  expected  discounted  income  using  policy  tt  and  starting 
in  state  s,  there  is  a  stationary  policy  tt  such  that  I  (tt  ,s,B) 
>  I(tt,s,B)  for  all  policies  tt  and  all  initial  states  s  e  S. 


Blackwell  proved  this  theorem  when  transitions  from  s  to  s'  occured 
at  times  t=l,  2,  ...  and  r(s,CL,s')  is  deterministic.  If  the  inter- 
transition  times  are  random  variables  and  the  reward  from  a  transition 
depends  on  this  time,  the  theorem  and  proof  are  valid  when  r(s,£L,s')  is 

4 

replaced  by  its  expected  value  r(s,&,s')  given  that  the  transition  time 
distributions  are  such  that  the  length  of  the  experiment  t,  and  the  number 
of  transitions  n,  have  the  property 


t  -•  ®  <=>  ri  -*  oo  a  ,s . 


If  the  transition  times  are  not  degenerate  at  ze«*o  or  defective,  a 

sufficient  condition  for  this  property  to  hold  is  that  there  is  a  finite 

number,  k,  of  transition  time  distribution  functions,  since: 

a)  If  t.  denotes  j-th  transition  time,  the  elapsed  time  after  n  transi- 
-*  n 

tions  is  t  =  E  t.  ,  and  n  <  ®  implies  t  <  ®  a.s., 

j  =  l  J 

by  f  orming  the  con tra -pos i t i ve  statement. 


sot-*®=£>n-«coa.s. 
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must  have  been  the  probability  law  for  a  minimum  of  -=  n1  transitions. 

n 1 

The  elapsed  time  for  these  transitions  is  t'  =  E  t,,  where 

i  =  l  ' 

t  j  ~  F  |  ( t) ,  and  n-*®=>n,-*“=>  t*  -♦  »  a.s.  =>  t  =£>  »  a.s. 

Since  our  problem  has  only  two  transition  time  distributions,  A ( t )  and 
B(t),  we  conclude  there  is  an  optimal  stationary  policy  when  future  costs 
are  continuously  discounted. 

2.3  Problems  Without  Discounting 

For  the  single  channel  stochastic  server  system  with  3=0,  we  estab¬ 
lish 

Theorem  2;  There  exists  a  stationary  policy  f  that  maximizes  the  reward 
rate,  and  f  is  independent  of  the  initial  state. 

I 

Proof:  First  we  show  that  there  is  a  stationary  optimal  policy^.  From  the 
preceding  theorem,  if  a  positive  interest  rate  were  used,  for  each 
initial  state  there  would  be  a  stationary  policy  f$  (3)  that  would 
earn  the  maximum  expected  discounted  reward  l[fs(8)]  =  I  (tt  ,s  ,0) . 

In  Section  III  it's  shown  that  the  holding  costs  are  propor- 

-2  -1 
tional  to  3  and  the  operating  costs  are  proportional  to  3 

This  implies  that  for  small  interest  reates  the  server  must  be 

turned  on  eventually,  i  .e . ,  there  is  a  finite  bound  on  n  .  Thus, 

there  is  an  interest  rate  3q,  a  sequence  (3  =  Cflj}*_Q  "*  0+  with 

8,  <  8.,  and  a  finite  set ,  $  -  {f  :f  =  f  (0.)}  ,  of  stationary 

I  *  0  5  5  S  S  I 


This  part  of  the  proof  is  based  on  the  work  of  Fox,  reference  8. 


T 
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optimal  policies.  Since  for  every  B.e  (B  there's  a  corresponding 
f  s ,  at  least  one  member  of  must  appear  infinitely  often  in  the 
sequence  {f  (B. This  implies  that  there  is  a  subsequence  of 

S  I  I  —  I 

=  f 3 j ^ j*_ |  “*  0+  and  a  stationary  policy  fg  that  is  optimal  for 
all  interest  rates  B^elS'. 

Using  standard  Abelian  and  Tauber ian  theorems  (reference 
Van  der  Pol  and  Bremmer  [15]  pg.  122), 


C  (tt)  =  lim  8 1*  e"^td  e?(TTc  ,t  ,0) 

B-»0+  J  S 

=  lim  81  (It) 

8-C+ 

<  lim  6.1  (f s )  =  C(f  ). 

j-*00 

The  result  that  the  optimal  stationary  policy  is  independent 
of  the  starting  state  follows  from  the  fact  that  introducing  any 
finite  cost  on  each  of  the  first  fi’nite  number  of  transitions  won't 
alter  the  asymptotic  cost  rate.  If  the  service  system  starts  in 

some  state  i  >  0,  it  will  incur  only  a  finite  expected  cost  before 

there  are  zero  customers  in  the  queue  and  the  server  is  idle,  so 

c(f.)  -c(f0).  QEd. 

The  above  theorem  was  proved  for  the  particular  case  of  interest  rather 
than  as  a  general  dynamic  programming  result  because  it  is  not  true  in 
general.  Suppose  the  state  space  is  $  =  (0,  1,  2,  ...),  and  in  each  state 

s  the  alternatives  are:  l)  go  to  s+1  in  one  time  unit  and  receive  no  pay¬ 

ment,  and  2)  go  to  state  s-0  in  s  time  units  and  receive  s  -  Any 
stationary  policy  of  going  to  state  zero  when  the  process  reached  state 


II 
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n  would  have  a  rate  of  r(l  -  -rr) ,  and  a  greater  reward  rate  can  be  achieved. 

Furthermore,  when  the  rewards  are  discounted  there  wi  1 1  be  a  stationary 
policy  from  theorem  1. 

Now  that  the  existence  of  stationary  optimal  policies  for  both  models 

j 

has  been  established.  It  is  possible  to  specify  the  form  of  such  policies.. 

Theorem  3:  Under  a  stationary  optimal  policy  either:  (1)  the  server  will 

always  be  dormant,  (2)  it  will  always  be  running,  or  (3)  it  will 
be  turned  on  when  there  are  n  customers  waiting  before  a  dor¬ 
mant  server  and  turned  off  when  the  system  enters  an  idle  period. 


Proof :  The  first  case  arises  only  when  discounting  is  used  and  the  holding 
cost  is  small  compared  to  the  operating  costs,  and  corresponds  to 
using  action  1  in  the  unprimed  states  so  that  the  primed  states  are 
never  reached. 

If  this  is  not  the  form  of  the  policy,  there  must  be  a  state 
s  where  action  2  is  optimal  (this  state  need  not  be  unique  as 
illustrated  in  example  3  of  Section  3.4),  so  s=n  .  It  remains  to 
be  shown  that  there  is  no  state  s'  in  which  action  1  is  optimal 
when  process  is  in  state  k'.  Since  this  state  will  be  reached  with 
probability  one,  the  process  will  alternate  between  k  and  s  cus¬ 
tomers  in  the  system,  once  the  server  is  turned  on,  which  is  equiv¬ 
alent  to  incurring  the  cost  of  reaching  state  s  and  then  holding  k 
customers  on  the  side  and  turning  the  system  on  when  s-k  are  present 
and  off  when  zero  are  present,  a  strategy  that  is  dominated  by 
serving  the  k  customers  (possibly  after  a  certain  finite  time  t)  and 
then  turning  the  system  on  at  s-k  and  off  at  zero.  If  k  >  s  and 
action  1  is  used  in  state  k',  the  equivalent  cost  is  achieved  by 
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reaching  state  s  and  then  holding  s  customers  while  turning  the 
server  on  at  k-s  and  off  at  zero,  which  is  dominated  by  serving  the 
s  customers  after  holding  them  no  more  than  a  finite  time,  and  then 
turning  the  server  on  at  k-s  and  off  at  zero.  QED . 

This  theorem  applies  only  to  the  allowable  decision  points,  specifically, 
only  to  those  points  when  the  server  is  busy  and  a  service  has  just  been 
completed,  otherwise  it  may  not  be  valid.  For  example,  consider  a  service 

{999  o=l  T 

*001  a-1000/'  Suppose  we  observe  the  system 

2  time  units  after  a  service  has  begun.  We  know  that  this  customer  will  not 
complete  service  for  another  998  time  units;  if  the  interest  rate  is  high 
enough  it  will  be  better  to  turn  the  system  off  and  incur  only  the  idle 
cost  for  a  while,  and  have  the  running  costs  charged  when  their  present  value 
is  lower. 

When.  0=0,  the  server  will  never  be  turned  off  when  customers  are  present 

4 

because  this  will  only  introduce  additional  on-off  costs. 
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Chapter  1 1 1 

THE  UNDISCOUNTED  INFINITE  HORIZON  MODEL 

In  this  chapter  we  will  present  two  ways  to  calculate  the  asymptotic 
cost  rate  (see  Section  2.1)  when  a  stationary  policy  of  activating  the 
server  when  n  customers  are  present  Is  employed;  we  then  derive  the  station¬ 
ary  optimal  policy.  In  Section  3.5  we  discuss  the  effects  of  a  proportional 
Increase  in  the  service  and  running  cost  rates;  in  Section  3.6  bounds  on 
the  cost  rate  and  optimal  policy  are  obtained  when  the  inter-arrival  times 
are  drawn  from  an  IFR  distribution. 


3.1  Derivation  of  the  Asymptotic  Cost  Rate.  Method  I 


Since  the  server  must  be  turned  on  eventually,  we  only  need  to  con¬ 
sider  policies  where  the  server  is  always  running,  or  is  turned  on  when 
n  >  1  customers  are  present  and  off  when  zero  customers  are  in  the  system. 
We  will  derive  the  cost  rate  for  the  latter  form  first. 

The  sequence  of  busy  cycles  forms  a  renewal  process,  and  a  basic  result 
of  renewal  theory  is  that  the  asymptotic  cost  rate  is  the  expected  cost  per 
cycle  divided  by  the  expected  cycle  time.  The  following  results  are 
immediate : 


(0 

(2) 

(3) 


1  .  VB 


The  expected  length  of  a  busy  cycle  is  n(^-  +  ) , 


The  expected  dormant  time  cost  in  a  busy  cycle  is  r-— , 

l 


nv, 


B 


The  expected  running  time  cost  in  a  busy  cycle  is  rj 


(4)  The  total  set-up  cost  in  a  busy  cycle  is  Rj  +  R£. 

To  calculate  the  expected  holding  costs  we  need  to  know  the  expected 
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value  of  the  sum  of  the  life-times  of  the  customers  served  in  a  busy  period; 
call  this  number  W^n^.  Consider  an  M/G/1  queueing  system  in  which  the 
server  is  always  running,  and  number  the  busy  periods  by  i  and  let  T| j  be  the 
number  of  customers  served  in  busy  period  i.  The  average  life  time  of  a 

customer  served  in  the  i -th  busy  period  is  Wj  =  (uuj^  +  (o^  +  + 

where  oj!  represents  the  life  time  of  the  j -th  customer  served  during 

'  '■  , 

busy  period  i.  Thus,  W.N.  =  £  oj,  and  multiplying  and  dividing  the  right- 

J  =  1  J 

n 

hand  side  by  E  T) f ,  dividing  both  sides  by  n,  rearranging  terms  and  taking 
i=1  1 


1 imits 


0) 


n  *« 


n  £  £  ttl 

I  n  .  |  |  |  J  »  n 

lim  1  E  w  1)  =  lim  ■  -  lim  -  E  1L. 

n-«»  n  1  =  1  n-«  ”  _  n-«  n  i  =  l 


E  T| 
i  =  l 


I 


Since  busy  periods  are  independent  and  identically  distributed,  the 
left-hand  side  of  (1)  is  a.s.  and  the  second  limit  on  the  right  is 

E(T)j)  a.s.  by  the  strong  law  of  large  numbers.  John  D.  C.  Little  [111 
showed  the  remaining  limit  is  the  expected  life  time  of  a  customer  W,  so 
WT  =  WE(T1). 

This  result  refers  to  a  busy  period  started  by  one  customer.  If  n 
customers  start  the  busy  period,  the  total  wait  is  the  same  if  we  serve  the 
first  customer  in  the  queue  first,  then  those  customers  who  arrive  during 
this  service,  then  the  customers  who  arrive  during  these  services,  and  so 
forth  unti  i  the  busy  time  generated  by  this  customer  is  completed,  at  which 
time  the  second  of  the  original  customers  starts  his  service,  and  the  pro¬ 
cess  is  repeated  until  the  queue  is  empty. 
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Since  the  arrivals  are  Poisson,  the  total  waiting  time  generated  by  each 
progenitor  are  independent  and  have  a  mean  value  of  W^..  While  the  first 
progenitor  and  his  descendents  are  being  served  n-1  progenitors  are  waiting, 
n-2  progenitors  wait  during  the  busy  time  generated  by  the  second  progenitor, 


and  so  forth.  This  con 


tributed  to  the  expected  total  waiting 


time.  During  the  idle  period  the  queue  is  building  up  to  n  customers,  so 


a<^e^  to  *^e  expected  total  waiting  time.  Adding  the  parts  we 


have 


u(n)  1"  V1G  A  n-1  ,  .  K~1 

t  *  +  —  <vig  +  r>J  "• 


Assembling  all  the  parts,  the  cost  rate  C(n)  is  given  by 


C  (r>) 


Rl+R2+  X  +  nrlVlG  +  b[ l-o  +  "2  ^V1G  +  xO  n 


n(X  +  VIG> 


which  simplifies  to 


C(n)  =  r^  +  (rj-r^p  +  hXW  + 


X(1-p)(R,  -  R2)  __ 


+  h 


Using  the  relationship  L  =  XW,  we  have  finally, 


,  x  (l-e)  (r  +R) 

(2)  C  (n)  =  r(  +  (r2-rt)p  +  h(L  +  -y4  +  - - - 
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3.2  Derivation  of  the  Asymptotic  Cost  Rate,  Method  II 

A  second  method  of  deriving  equation  (2)  is  to  observe: 

(1)  the  running  cost  of  the  server  can  be  considered  as  an  increment 
of  r2"r]  over  a  constant  cost  of  r^  and  that  this  increment  is 
incurred  a  fraction  o  of  the  time,  independent  of  n,  since  p  is 
the  probability  that  the  server  is  operating  at  an  arbitrary  point 
in  time  (Theorem  1-4) 

(2)  Rj  +  F*2  is  incurred  during  each  busy  cycle  and  from  the  elementary 

renewal  theorem  the  number  of  busy  cycles  per  unit  time  is 

1  /I  .  x-1  X(l-o) 

n  <X  +  V  =  "lTtL;  and 

(3)  the  holding  costs  per  unit  time  are  proportional  to  average  number 
of  customers  in  the  system. 

The  average  number  of  customers  in  the  system  is  L^,  which  can  be 

4 

expresses  as 


=  [l_^|  Server  Busy]P  (Server  Busy)  +  [L  ^  |  Server  Id  le]P  (Server  Idle). 

The  average  number  of  customers  in  the  system  when  the  server  is  idle 
is  and  from  the  arguments  used  to  derive  w|n^  it's  easy  to  see  that 

the  average  number  of  customers  in  the  system  when  the  server  is  busy  Is 
Therefore, 


See  Barlow  and  Proschan  [11,  Theorem  2.5,  for  a  precise  statement  and 
proof . 
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(3)  L<">  =  (L<‘>  ♦  Bji)p  +  BjL  (1-p)  =  L  +  Bjl. 

and  thus  we  obtain 

(R.  +  R,)X(l-p) 

C(n)  =  r,  +  (r2-r,)p  +  h(L  +  +  - - - , 

which  Is  equation  (2). 

It's  important  to  note  that  equation  (2)  only  holds  when  n  >  );  if 
the  server  is  always  running,  there  is  no  idle  time,  start-up,  or  shut¬ 
down  costs,  so  the  cost  rate  is 


(4)  C  (0)  =  r2  +  hL. 

3.3  Determining  the  Optimal  Policy 
Setting  =  0  we  obtain 

(5) 

(6)  C(n*)  =  rj  +  (r2-r?)p  +  h(L  -  J)  +  [2X(Rj  +  R2)h(  1 -p)l2. . 

d2*  /  \  ^ 

Since  - s— ^  >  0,  n  gives  the  unique  minimum  value  of  C(n)  for  1  <  n  <  ®. 

dn 

The  reason  that  n  doesn't  depend  on  r^  and  r2  is  because  for  all  va  1  ue s 
of  n  >  I,  the  fraction  of  time  the  server  is  busy  is  p  (Theorem  (-4);  the 
optimal  value  will  only  balance  the  increase  in  holding  costs  with  the 
decrease  in  set-up  costs  as  is  increased.  However,  the  optimal  policy 
depends  on  r^  and  r2  because  equation  (4)  may  show  a  cost  rate  lower  than 
the  one  given  by  equation  (6). 


'2\{K 


n  = 


+  R2)0-p) 

n 
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The  optimal  policy  must  be  given  by  an  integral  value  of  n,  call  it 
.  If  n  is  bigger  than  one  but  is  not  an  integer,  the  best  Integer 
value  of  n  is  one  of  the  integers  surrounding  n*;  therefore,  n°pt  is  either 
one  of  these  integers  or  zero.  The  decision  is  made  by  evaluating  the  cost 
rate  given  by  each  of  the  three  candidates. 

If  0  <  n*  <  1,  either  nopt  =  0  or  n0pt  =  1;  since  C(0)  <  C(l)  when 
(r2-r,)  (1-p)  -  (R !  +  R2)X(1-p)  <  0,  or 

(7)  r2-r,  <  X(R,  +  R2) 

so  the  choice  is  easily  made. 

When  n*=0,  either  X=0  or  Rj  +  R^  =  0  so  n0^  =  0.  Summarizing,  n0*** 
is  either  0,  [n*],  or  [ri*  +  1]. 

3.4  Numerical  Examples 

Example  1;  p=^,  X=l,  h=l ,  Rj  +  R2  =  4,  r  p ! ,  r2=4 

Then  n*  =  1  X  4  X  \  =  2,  and 

C  (2)  =  1  +  3  X  i  +  (L  +  1)  +  4  X  £  f  2  =  4J  +  L,  and 
C(0)  =  4  +  L. 

Therefore,  the  optimal  policy  is  to  leave  the  server  on  all  the 
time. 


Example  2:  p=r|,  X=1 ,  h=l,  Rj  +  R2  =  5,  r pi ,  r2=6 

Then  n*  =  X  2  X  5  =  5  «  2.2,  and 
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C  (0)  =  6  +  L, 

C(2)  =  1  +  £  +  (L  +  i)  +  5  X  $  f  2  =  5i  +  L. 

C(3)  =  l+  i+(L+l)+5Xjr3  =  5T+L. 

Therefore,  the  optimal  policy  is  to  turn  the  server  on  when  2 
customers  are  present. 

Example  3:  p=£,  X=l,  h=2,  R^  +  R2  =  4,  r ^  —  1 ,  r2=5 

Then  n*  -Jl  X  |  X  4  =  2,  and 

C(0)  =  5  +  2L, 

i 

C (1)  =  1+  2  +  2L  +  £  X  4  «  5  +  2L, 

C (2)  =  1  +  2  +  2(L  +  £)  +  £  X  4  t  2  =  5  +  2L. 

Therefore,  there  are  3  optimal  policies. 

3,5  A  Control  Model 

We  have  been  considering  the  service  time  as  a  random  variable  a  with 
d.f.  6^(0 .  The  random  varialbe  a1  =  0ct,  which  is  a  mapped  onto  a  different 
time  scale,  has  the  properties: 

(D  Bol(t)-B0$. 

(2)  E(o')  -  6E (a) 


When  0  <  1,  a'  is  stochastically  smaller  than  o  so  this  can  be  inter¬ 
preted  as  an  improvement  in  the  service  facility;  similarly  0>  1  represents 
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a  decrease  in  the  capability  of  the  server. 

Suppose  we  can  change  o  to  o'  with  a  corresponding  change  in  running 

r2 

cost  from  to  rJJ  =  — ,  with  all  other  costs  remaining  the  same.  In  the 

undiscounted  case,  the  optimal  policy  with  respect  to  opening  the  server 
becomes 


(8) 


+  R2)X(l-0p) 
- 


since  p'  =  XE  (0a)  =  9p .  Since  the  optimal  policy  for  serving  customers  is 
uniquely  determined  by  equation  (1),  the  cost  rate  can  be  expressed  as  a 
function  of  8  only,  VIZ.  , 


(9)  C  (0)  =  r .  (1  -0p>  +  r.p  +  h[L(0)  +  n  +  (R  +  R  ) 


0o_ 


2'  n*(0) 

02p2(Cg  +  1) 

where  L(0)  =  0p  +  — - from  the  Pol  laczek-Khinchin  equation. 

One  can  find  the  optimal  value  of  0  by  investigating  the  roots  of 


(10) 


8p2(r^  +  l)(2-9p),  1  i 

+  - - - 5 - -  p[2(1-8p)?%h(R,  +  R,)]*  +  r 

2(l-8p)  J  ' 


=  0. 


A  solution  to  equation  (10)  can't  be  given  in  closed  form;  however, 
one  can  obtain  the  conditions  for  which  it's  desirable  to  slow  the  server 
down.  The  initial  control  setting  is  9°=1;  C  (0)  decreases  as  0  increases 
when 

^  1 8=1  ■  +  P  (22~(!5— ]  -  +  pMki-p)]-*  -  r,p  <  0. 
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Simplifying,  we  have  the  local  condition  for  slowing  the  server  dwn, 


00 


rX(R,  +  R 2Ul  P  (2-p)  (Cg  +  1) 

I  2h(T-p)  J  +  IT  >  1  +  ^(WO  ; 


if  the  reverse  inequality  holds,  the  server  should  be  speeded  up. 

The  qualitative  reasoning  that  explains  the  form  of  this  result  is  that 
increasing  the  service  time  will  lengthen  the  busy  periods  and  reduce  the 
fraction  of  time  the  server  is  idle,  which  is  desirable  if  Rj  +  R2  or  r^ 
is  large.  It  also  has  the  effect  of  increasing  the  mean  number  of  customers 
in  the  system,  which  is  undesirable  if  h  is  the  dominant  cost. 


3.6  General  Arrival  Distributions 

Equation  (2)  is  not  valid  for  G/G/l  queues  because  4  n v ^ ^ ;  however, 

n 

we  expect  that  equation  (5)  will  be  approximately  correct  when  p  is  close  to 
unity,  giving  a  policy  that  in  a  highly  utilized  system  the  server  should 
always  be  available,  unless  the  holding  costs  are  negligible.  For  arbitrary 
values  of  p  we  can  obtain  bounds  on  the  cost  rate  and  optimal  policy  if 
the  arrival  distribution  is  suitably  restricted.  Generalizing  the  arrival 
distribution  requires  that  the  decision  points  during  an  idle  period  be 
restricted  to  arrival  epochs,  as  pointed  out  in  Section  2.1. 

A  class  of  distributions  that  has  been  given  much  attention  in  reli¬ 
ability  models  is  the  class  of  IFR  distributions,  where  IFR  stands  for 
increasing  failure  rate.  The  IFR  assumption  for  A(t)  corresponds  to  arrival 
patterns  where  the  probability  that  a  customer  arrives  in  the  next  time 
increment  increases  as  the  interval  since  the  last  arrival  epoch  increases; 
imposing  the  IFR  assumption  on  the  arrival  stream  is  natural  for  models 
where  the  customers  represent  failed  pieces  of  equipment  and  the  service 
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mechanism  is  the  repair  facility.  Since  t':e  Erlang  distributions  and 
many  other  common  distributions  are  IFR,  assuming  A (t )  is  IFR  is  a  non- 
parametric  way  of  analyzing  a  large  family  of  common  queueing  models. 

A  distribution  A ( t )  is  IFR  if 


A(t  +  x)  -  Aft) 
A°(t) 


is  increasing  in  t  for  all  t  >  0;  the  properties  of  these  distributions  are 
discussed  in  Barlow  and  Proschan  [I]. 

Marshall  [12]  found  bounds  for  the  expected  number  of  customers  In  the 
system  and  the  expected  length  of  the  busy  cycle  for  the  Gn/G/1  queue  we 
shall  consider.  When  A(t)  is  IFR  and  p  <  1 ,  he  obtained: 


(12) 


A<L<  +  ^,  A=p 


p[c*-l]  +  pVB  +  I] 

zTwJ 


+ 


2  2 

where  and  are  the  coefficients  of  variation  of  A (t )  and  B(t)  respec¬ 
tively,  and 


03) 


ikplfcrp-l  <  e  (x)  < 


where  E(X)  is  the  expected  length  of  a  busy  cycle. 

Since  the  cost  rate  is  highest  when  L  .  sumes  its  largest  value  and 
E(X)  Its  smallest,  and  the  minimum  cost  rate  is  achieved  when  these  con¬ 
ditions  are  reversed,  the  cost  rate  is  bounded  for  all  n  by 


(R,  +  R2)X 


r ,  ( 1  -p )  +  r 2p  +  hXA  +  - -  C  (0)  <  r^l-p) 


(14) 


(R1  +  R2)X 


+  r2p  +  h(\A  +  |)  +  n  >  I 


[< 


r2  +  |  P  + 


P[c^-n  ^  p2[c^  +  n 

2(l-p) 


•]  <  c  (0) 


<  r. 


r  p[C?-l]  +  P2[C^  +  I]  , 

+  i* +  -~2(i-p> — +  *]■ 


The  width  of  the  bounding  interval  is  easily  calculated  to  be 

ft,  +  h 

— rr '  v  +  V,  n  >  1,  which  decreases  as  n  increases  and  p  decreases. 
-p)(n-p)  r  K 

Let  Cu(n)  and  (n)  be  the  upper  and  lower  bounds;  simple  differentiation 

reveals  that  both  are  convex,  so  C(n)  is  bounded  by  an  interval  as  shown 

1  n  Figure  1 


Setting  the  derivative  of  Cu(n)  with  respect  to  n  equal  to  zero  yields 


(15) 


*  P(R  t+R2> 

n  = 
u 


'I  L'  w 

hTT^ol —  +  p  >  "  1 


where  n  is  the  optimal  policy  when  the  arrivals  are  Poisson  (equation  5), 
and 


(16) 


*  n  ptc.“i]  +  p  [c_  +  i]-. 

cu(nu>  =  ri +  <r2-ri>p +  h[i p  + - rfi”py  ® — ] 


[ 


2X(R.  +  R,)h,| 


1  '  "2JK1* 
l-p  J  ' 


Since  n  obtains  the  minimum  value  of  the  maximum  cost  rate,  it  is  the 
u  * 

minima*  t  +  rategy  and  Cu(n'u)  is  the  upper  bound  for  the  optimal  cost  rate. 
In  a  similar  manner,  one  finds  that  Cj  (n)  is  minimized  by 


(17) 


*  / 
ni  */■ 


2(R,  +R2)X  * 

hTw5 — >n 


and 


(18) 


C,  ("?> 


r  P(C.“1)  +  p  [C.  +  1)  -i 
“  r,  ♦  (r2-r,)p  +  h|_p  + - - ij 


p2X  (R  |  +  ^2^ 

+  L"”  n;  j  • 


Since  Cj  (n t )  is  the  lowest  possible  cost  rate  and 
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the  optimal  cost  rate  is  bounded  within  j  of  its  value  if  A(t)  was  com 
pletely  specified. 

From  the  diagram  it  is  easy  to  see  that  the  values  n^  and  n^  that 
satisfy  C^(n^)  =  C^n^)  =  C^n^)  are  the  bounds  on  the  optimal  policy. 

Using  equations  (14)  and  (16)  one  obtains 


(20a)  nj  =  nu  +  l-2p  -  [(1-p)  +  4j^ - - - jj 


*  r  2  r^^(Ri  +  “P  )"l«n  2 

(20b)  n2  =  nu  +  1  -2p  +  [(1-p)  +  4[ - ^ - ■]  J 


(20c) 


n2 


«  2 


where  n  Is  given  by  equation  (5).  Since  small  values  of  h  cause  large 

ic 

values  of  n  ,  equations  (19)  and  (20c)  imply  that  as  the  bounds  on  the 
optimal  cost  rate  get  better,  the  bounds  on  the  optimal  policy  will  get 
worse.  This  means  that  large  variations  in  the  policy  will  have  small 
effects  on  the  expected  cost  rate. 


3.7  Conclusions 

From  equations  (6),  (16),  and  (18)  we  see  that  the  expected  cost  rate 
can  be  reduced  by  decreasing  the  variability  of  the  service  and  inter-arrival 
time;  thus .  ce  terus  paribus,  the  queue  with  constant  inter-arrival  and 
service  times  will  have  the  smallest  cost  rate. 

|f  we  assume  a  Poisson  arrival  stream  but  A(t)  is  actually  IFR,  sub¬ 
stituting  equation  (5)  into  the  equation  for  cu(n),  we  find 
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(20) 


C  (n*)  -  C  (n*) 
u  u  u 


-(R|  +-"-2>  .  l!_?_elh  („*.!). 
(l“p)(n  -p) 


Since  small  operating  costs  and  large  holding  costs  tend  to  give  the 

policies  n°*,t  equal  zero  or  one  for  both  Poisson  and  IFR  arrivals,  the 

*fc 

error  given  by  equation  (2)  will  be  large  only  when  n  is  large.  Thus,  when 
the  utilization  and  holding  costs  are  large  and  the  operating  costs  are 
small,  the  loss  in  generality  in  assuming  Poisson  arrivals  Is  not  very 


critical . 
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Chapter  IV 

THE  D l  COUNTED  INFINITE  HORIZON  MODEL 

In  this  chapter  we  will  find  the  expected  present  value  (discounted 
total  return)  of  a  stationary  policy  that  turns  the  server  on  when  n  cus¬ 
tomers  are  present,  and  turns  it  off  when  the  system  is  empty.  The  analysis 
in  this  chapter  is  more  complicated  than  that  of  Chapter  III  because  explicit 
account  must  be  taken  of  the  epochs  when  costs  are  incurred. 


4.1  Some  Properties  of  Laplace-Stieltjes  Transforms 

We  will  use  the  following  properties  of  Laplace-Stieltjes  transforms 
extensively  in  this  section: 


Property  1:  If  a  cost  R  is  incurred  t  time  units  from  now,  and  t  is  a 

random  variable  with  d.f.  F(t),  then  the  expected  present  value 
of  this  cost  is  rF(8). 


•8 1 

Proof:  Assume  that  T=t,  then  the  discounted  cost  is  Re  ,  unconditioning 

on  T=t  we  have  the  expected  cost  is  [  Re  BtdF(t)  =  R^(B),  QED. 

*0 

Property  2:  If  a  cost  rate  of  r  dollars  per  unit  time  is  incurred  until  t 
units  from  new,  and  t  is  a  random  variable  with  d.f.  F  (t)  ,  then 
the  expected  present  value  of  the  cost  is  ^-[1-F($)]. 

**8 1 

Proof;  Assume  t  >  t,  the  present  value  of  the  cost  at  time  t  is  re  .  The 

probability  that  t  >  t  is  l-F(t)  so  the  expected  present  value  of  the 

cost  at  time  t  is  I  re  ^[l-FftJjdt  =  r-  -  rf  e  ^  tF  (t )  dt  =  l -F(8 )  1 , 

J0  B  J0  P 

QED. 
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Property  3:  If  F (t )  is  a  distribution  function  of  a  non -negative  random 
variable  that  is  not  degenerate  at  the  origin,  then  if  F(9)  - 

f  e  ®tdF (t)  exists,  0  <  F(P)  <  1  for  p  >  0. 

J0 

u  /» 

P roof :  f^p)  >  0  since  e  Pt  >  0  and  dF (t)  >  0  but  not  identically  zero. 

?^(P)  <  1  since  |  J"e”BtdF  (t)  |<|e~r  "j  |  J  dF  (t )  |  <  1. 


4,2  Calculation  of  the  Expected  Total  Cost 

Using  these  results,  we  first  find  the  expected  cost,  C(n;P),  for 
n  >  1,  starting  with  zero  customers  in  the  queue.  The  first  two  properties 
show  that  in  the  first  busy  cycle: 

(1)  the  expected  start-up  cost  is  Rj[^(P)]n  =  ”  cj 

(2)  the  expected  shut-down  cost  is  R2[tf(P)G(P)]n  =  R2  “  c2 

(3)  the  expected  idle  time  cost  is  ~  [  1  -(^(B))n]  =  jp  [*  1  -  °  J  “ 

(4)  the  expected  running  cost  is  ^  [  1 -[£($)  (3) J  =  jp  (p^)° 

[i-(g'(p))n]4c4 

The  LST  of  a  busy  cycle  is  Wn(P)  =  [ft(p)*£(p)]n ,  so  '.he  expected  dis¬ 
counted  operating  costs  for  the  entire  planning  period  is 

(1)  0(n;P)  =  c  +  cH  (P)  +  c[Hn(P)]2  +  ...  =  Vq\ 
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where  c  =  Cj  +  r  r3+c4»  anc^  0  <  Hn(3)  <  1  from  property  3. 

The  holding  costs  will  be  calculated  in  two  parts.  When  the  queue  size 
is  increasing  from  zero  to  n  during  an  idle  period,  one  customer  waits  for 

n-1  arrivals  which  cost  g-ft(8)  !" 1  -  [^(3)ln""^ 

Inter-arrival  time  from  the  beginning  of  the  busy  cycle.  The  second  cus¬ 
tomer  arrives  after  two  inter-arrival  intervals  and  waits  for  n-2  more 

arrivals,  so  the  holding  cost  is  g* 

remaining  customers  is  computed  in  the  same  way,  and  the  total  discounted 
cost  is 


[*(B)]2[l  -  [fftB)!"-2];  the  costs  of  the 


J  since  he  starts  waiting  one 


Vn:B)  =  Hy(B)CI‘<ff(8))n"l]  +  We)]2[^(B))n'2l  +  ••• 

+  (*'(e))n'1[i-K(e)] 

h^(B)  -  y(B)j"  ,  (n,l)(y(e))ni 

p  I  -  K(B)  J 

(2)  ,  (n;B)  .  h  K(B+X)n  -  xn.(xinel 

q  8  B(B+X)n 

The  expected  discounted  total  cost  over  an  infinite  horizon  is 

1  (n-B)  h  X(B+X)n  -  xn<X+nB) 

q  l4Tn(B)  B2  (6+X)n  -  [X*(8)]"  ' 

The  remaining  costs  are  due  to  customers  who  are  waiting  while  the 
server  is  busy.  If  tg(n;T)  is  the  expected  number  of  customers  in  the 
system  at  time  t,  and  t  is  a  point  in  the  first  busy  period,  the  expected 
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discounted  cost  of  holding  these  customers  until  t  +  dT  is  he”^7ts (n ,T)dT 

and  the  cost  for  the  busy  period  is  l$(n;p)  =  hj  e"^Tl  (n ;T)dT,  where  t  Is 

J0  S 

a  random  variable  with  d.f.  G  (t). 

Let  0  equal  the  service  time  of  the  first  customer  served  in  a  busy 
period  and  k  be  the  number  of  arrivals  during  0,  then 

(**)  I  (1  ;B| 0,k)  .  hf  e~^Tt  (1  ;t| 8,k)dT  +  e'88l  (k;8) 

Jo  s  s 

=  h f  e  ( 1  +  kt)dt  +  e“B0l  (k;0). 

J0  s 

Changing  the  order  of  service  as  in  Section  3,  and  using  properties 
one  and  two,  we  find  that 


1  (n;8)  =  *,(1  ;B)C  I  +  5(8)  +  (5(8))'  +  ...  +  (G(8))n"'l 


(5) 


+  h  !^-[l-5(8)1  +  h  1-5(8)15(3)  +  ...  +  h  £[1-5(8)! 


n^2 

8 


[G(B)!"-2  =  .  0;B)  H2feli:+&r»  - 

5  I -5(B)  8  -  1-5(8)  J 


Substituting  equation  (5)  into  equation  (4)  and  unconditioning  on  k, 


t  (1  ;B  |  0)  =  hf  e”0*  (1+Xt)dt  +  e"B0  (l  (1;B)  -X9(lJ^(6))'1 

*  Jo  L  5  1^5(8) 


+  h|\e  l-expr-X»(l-5(8))TTj 
8  1-5(8)  J)' 


integrating  the  first  term  by  parts  and  simplifying  the  second  term  gives 
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(6)  isO;e|e) 


-Be 


-l)h  + 


ileJS£[-\Ql)-G(Q.)).l  [t  (,.p) 
1-^(6) 


Unconditioning  on  6  and  reca  1 1  i  ng  Gf(3)  =  BtS+X-XG^B)]  , 


ls(l;B)  =  ~  h  +  [  t  $  ( 1 ;  3 ) 

P 


B(P)-G(B) 

l-G(B) 


from  which  one  obtains 


(7)  I  (l;B)  -  —  [  l-'g'{8)lh  +  51SI3M  h. 


Substituting  equation  (7)  into  equation  (5),  the  expected  cost  of  a 
busy  cycle  is  seen  to  be 


(8) 


ls  (n;B) 


n-(G(B»nirX-(XH-B)B(R)1h  +  nh 

e2[i-B(en  B 


when  h=1,  l$  (n  ;B)  is 
the  sys  tem  dur  i  ng  a 
is 


the  Laplace  transform  of  the  mean  number  of  customers  in 

•is* 

busy  period  .  The  expected  costs  for  the  entire  horizon 


(9)  Ls(n;8) 


‘s(n;e)  n-(G(n))nirx-(x+Bi?(Bnh 

B2[l-B(B)][1-(*f3p)n1 


Thus,  the  total  expected  discounted  cost  function  is,  finally: 


This  transform  can  be  computed  from  the  results  of  Gaver  [91  but  this 
derivation  is  more  direct. 
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(10)  C(n;B)  =fl(n;fi)  +  L^(n;8)  +  Ls(n;B) 

PR,  +  B[G(6)]nR2  +  (r2-r,)[l-(ff(B))n]  r, 
n|-(X+B)ff(pn  B 

+  h_  X(X+B)n-Xn(X+nB) 
e2  (S+X)n-  [XG(8)ln 

+  ri-(G(e))nirx-ftt6)B(B)ih 
B2[i-e(B)l[i-(^M)n] 


if 

In  principle,  one  can  find  the  minimizing  value  of  r,  n  (@)  by  classical 
calculus  methods;  however,  a  closed  form  solution  for  the  usual  first  order 
conditions  doesn't  exist.  Equation  (10)  contains  terms  that  are  linear  In  n 
and  terms  in  which  n  is  an  exponent,  hence,  the  derivative  of  O  ;3)  with 
respect  ton  will  also  contain  terms  of  these  forms.  This  implies  that 

^0r(n;6)  =  0  is  an  implicit  equation  in  p,  so  there  is  no  closed  form 

expression  for  its  roots.  f 

if 

Thus,  an  indirect  method  for  calculating  n  (0)  is  required.  For  example, 
a  computer  solution  using  successive  approximations  could  be  used;  for  small 
values  of  3,  one  could  approximate  ^-l^(n;8)  by  a  Haclauren's  series  expansion. 

In  the  discounted  case  both  endpoints  must  also  be  considered  as  pos¬ 
sible  minimizing  values  of  n.  When  n=0,  Rj  is  charged  when  the  first  cus¬ 
tomer  arrives,  the  running  cost  is  always  charged,  and  there  are  no  customers 
who  wait  for  the  server  to  be  turned  on,  so 


t 


I 
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R  |X 


do  e (o;s)  =  six  +  <r +  h  [t  + 

B X  B  V  R[s+X-XG(R)1[  l-'B(e)! 


} 


The  interpretation  of  n  =  ®  is  that  the  server  is  never  turned  on.  In 
this  case,  the  only  costs  incurred  are  holding  costs  in  the  queue,  and 


(12)  £(0;B)  =  lim  L  (n  ;B) 

n-*oo  H 


We  note  that  the  undiscounted  cost  rate  [Equation  (2),  Chapter  III] 
depended  only  on  the  first  and  second  moments  of  the  service  distribution, 
while  (n;6)  in  (10)  depends  on  the  knowledge  of  the  entire  distribution, 
through  IT (8)  and  Gf(8). 


4.3  Limiting  Results  When  the  Interest  Rate  Vanishes 

As  6  vanishes,  the  expected  cost  given  by^,(n;R)  approaches  infinity; 
an  important  result  is  that  as  8  approaches  zero,  the  discounted  model  gives 
the  same  cost  rate  as  the  undiscounted  model. 

Theorem  1;  Lim  p£(n;8)  =  C(n),  where  C  (n)  is  given  by  equation  (1 1 1 -2). 
8-0+ 

Proof ;  Recall  that  e.<  n;8)  and  C(n)  were  defined  in  Chapter  II  as: 

£(n;B)  =  f°e  8tdC(n,t),  C(n)  =  lim  7  [  d£(n,t),  where  C(n,t)  is 
J0  T-*®  '  J0 

the  cumulative  costs  incurred  at  time  t  using  a  policy  that  turns 
the  server  on  when  n  customers  are  present.  Since  C(n,t)  is  a  mono¬ 
tone  function  because  all  the  costs  are  non-negative,  we  can  apply 
a  standard  Tauberian  th-orem  for  LST's  and 


37. 


Lin  $£(n;fi)  =  Lim  p[  e  ^td(3(n;t)  =  Lim  =r  [  d£(n,t) 
B-0+  p-0  J0  T-to  1  J0 


=  C(n),  QED . 


*5f  «jj» 

Theorem  1  doesn't  necessar i  1  y  imply  that  n  (8)  -»  n  as  6  -•  0  because 
examples  have  bee.1;  constructed  for  other  models  (see  for  example  [2*])  where 
the  discounted  cost  function  satisfies  Theorem  l  but  the  policy  for  small 
interest  rates  is  not  "close"  to  the  undiscounted  policy.  However,  for  our 
problem,  it  js_  possible  to  show  that: 

& 

Theorem  2;  n  (p)  -  n  as  P  -  0. 


Proof :  Since  n  is  treated  as  a  continuous  variable  over  the  range  [  1 ,») 

J  J 

the  derivatives  n;B)  and  -r-  C(n)  exist  over  ( 1  ,*»)  ,  and 


lim  P6(n;P)  =  C(n), 
8-0 


(13) 


lim  p  ^C(n;p)  =  ^  lim  B£(n;0)  =  ^  C(n). 
P— 0  B—  0 


The  meaning  of  equation  (13)  is  that  there  exists  a  small 
positive  number  which  may  depend  on  P  and  n,  e(8,n)  say,  such  that 


(14)  B  -£-(:(n;p)  =  ^  C(n)  +  e(0,n) 


and  e(p,n)  -  0  as  B  -  0.  Pick  a  small  value  of  P  greater  than  zero, 
0°,  then  equation  (14)  implies 


(n;B°)  +  =(8°,n)] 

P 
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*Jf  Q 

and  when  n  =  n  (fi  )  , 


05)  -  ?[5n  C(n^n=n*(B°)  +  e<B°-"]  “  °- 


Since  —  4  0,  the  term  in  square  brackets  must  equal  zero, 
0° 

from  equation  (I  II -2)  means 


(R1  +  R2)\(l-p)  h  +  2e  (B°.n) 

A  *  1  +  o  -  0. 


[n*(B°n2 


This  implies 


(16) 


*/„Ox  /2X(R|  +R2^1_p^  *  „o  . 

n  (3  )  =  / - - - *  n  as  B  -  0. 

h  +  2e(8°,n) 


It  remains  to  be  shown  that  n  (B)  is  close  to  n  for  small 
values  of  B.  We  have 


r  *,2  2RX(R(  +  R2)(l-p)  h.+  2e(B.n)  . 

[n  ]  =  - E -  h  +  2e  (B  ]n)  a"d 


*,..,2  2X(R,  +  R2)0-p)  , 


[n  (B)l  = 


h  +  2e (B ,n)  h 


from  which 


(17) 


-*»)  -  "* 


The  square  root  term  can  be  made  arbitrarily  close  to  unity  by 
picking  6  close  enough  to  zero,  QED  . 


wh  ich 
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Equation  (17)  shows  that  convergence  is  faster  as  h  is  larger.  This  is 
because  for  small  values  of  h,  n  is  very  large,  but  if  the  interest  rate  is 

•jjf 

large  enough  n  (P)  is  infinite  (i.e.,  never  turn  the  server  on),  and  as  the 
Interest  rate  is  decreased,  n  (p)  decreases  and  the  approximation  of  n  (P) 

■ft* 

by  n  becomes  useful. 

The  operational  implication  of  Theorem  2  is  that  for  small  values  of 
P,  one  can  use  n  ,  which  is  easy  to  compute,  to  estimate  the  integer  value  of 
n(6)  that  minimizes  B(  n;P).  As  before,  this  integer  value  is  one  of  the 
integers  surrounding  n  (6),  if  n  (p)  is  not  an  integer,  because  of 

Theorem  3:  For  B  sufficiently  small,  £(n;P)  is  convex  in  P. 

Proof;  Differentiating  both  sides  of  equation  (5), 

2  2 

1  im  B  C  ("  *,8)  =  C  (n) 

P~*0  dn  dn 

which  means  there  exists  a  function  6(P,n),  such  that  for  small 
values  of  P 

2  2 

(18)  8  -  ^Cfn)  +  6(B.n)  =  +  6(8>n), 

dn  dn  n* 

and  6(P,n)  can  be  made  as  small  as  desired  by  picking  B  sufficiently 

close  to  zero.  In  particular,  since  we  don't  know  the  sign  of 

6(p,n),  P I  ck  p  small  |  6  (6  ,  n )  |  <  this  insures  that  the  right 

n 

hand  side  of  equation  (18)  is  positive,  and  since  P  >  0, 

j2 

S-ygfnje)  >  0,  QEO . 
dn 


The  above  development  has  assumed  that  the  system  is  initially  empty, 


and  the  discussion  in  Section  II  indicated  that  the  stationary  optimal 

/ 

policy  may  be  dependent  on  the  initial  state.  If  there  are  m  customers 
present  initially,  the  total  cost  function,  call  it  £m(n;B),  can  be  con¬ 
structed  from  our  present  results. 

Since  the  policy  is  assumed  stationary  when  we  reach  a  regular  busy 
cycle,  the  server  will  not  be  turned  on  until  n  customers  are  present;  if 
m  <  n ,  the  server  will  be  turned  on  after  n-m  arr iva  Is  ;  if  m  >  n ,  the 
server  will  be  turned  on  immediately.  Thus,  if  k  =  Max  [0,m-n]  is  the  first 
time,  and  n  is  the  number  present  when  the  server  is  turned  on  thereafter, 
we  can  express  the  expected  discounted  cost  function  in  terms  of  k  and  n, 
and  then  look  for  the  minimizing  values. 

Since  there  will  be  zero  customers  in  the  system  at  the  end  of  the 
first  busy  per  iod ,  ^(k  ,n  ;B)  equals  the  cost  of  the  first  busy  period  plus 
the  present  value  of  (?0(n;3)  started  after  k  customers  have  arrived  and 
the  first  busy  period  is  completed,  thus 

os)  eB(k.«i«  ■*,^g)k  +  i«2(^r)W)rk 

+  r['-<&>k]  +  r  (^)k['-(5(B))m+k) 

+  (g^)k[G(B))k+neo(n;8). 

Minimizing  values  must  be  found  numerically  or  by  some  approximation  tech¬ 
nique. 

Equation  (19)  indicates  that  the  optimal  policy  will,  in  general,  depend 


Ls(n;B) 

+  "i-OeT 
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on  on.  However,  as  g  -*  0+  8  m(k,n;B)  -*  C(n),  and  the  optimal  policy 

becomes  independent  of  m. 


4.5  An  Alternate  Formulation 

In  many  applications  the  value  of  h  is  not  known,  e.g.,  t'.e  holding 
cost  of  military  equipment  in  a  repair  depot,  and  one  might  formulate  the 
problem  as  minimizing  operating  costs  subject  to  some  operating  constraints 
such  as  the  expected  life  time  of  a  customer.  The  next  theorem  shows  that 
this  minimization  wi  1 1  be  trivial. 


Theorem  4:  The  operating  cost  Q (n ; B )  is  a  strictly  convex,  monotone 
decreasing  function  of  n. 


Proof:  When  0=0  this  theorem  refers  to  the  operating  cost  rate,  which  is 

X(l-p)(R,  +  R2) 

Tj  +  (r2-rj)p  +  — - -  ••  which  is  obviously  strictly  convex 

and  monotone  decreasing  in  n  (we  ignore  the  completely  trivial  case 
where  Rj  =  R^  =  0,  in  which  case  the  function  is  constant).  The 
case  where  0  >  0  is  more  complicated.  For  notational  simplicity, 
let  x  =  *B(0) ,  y  =  ^(8) ,  and  z  =  xy,  so 


(20) 


_  n  ,  _  n 
r.  R.x  +  R_z 

O(n;0)  =  ^  + 

B  1-z" 


Vrl  xn-zn 


l-z' 


and  0<x<  1,  0<y<  1,  0<  z<x,  0<z<y,  with  xn,  y°,  and  zn 
strictly  convex  monotone  decreasing  functions  of  n. 

The  first  term  of  equation  (20)  is  constant;  the  second  term 
is  obviously  decreasing  and  convexity  is  shown  by  writing  it  as  two 

terms  R.xn(l  +  zn  +  z^  +  ...)  +  R_zn(l  +  zn  +  z  n  + 


...)  which  is 
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a  non-negative  sum  of  convex  terms.  The  third  term  is  a  constant, 


r2"rl  xn-zn 

— =■— -  multiplied  by  the  function  -  =  f(n). 

8  l-zn 

The  proof  of  the  theorem  is  completed  by  proving  f(n)  is  mono¬ 


tone  decreasing  and  strictly  convex,  which  we  do  by  induction.  Let 
n=l;  the  first  difference  is 


2  2 

X-Z  X  -z 
'**  '  l-z2 


l-z2)(x-z)-(l-z)(x2-z2] 
(l-z) (l-z2) 


[x-z)(l-z)(l-x] 
(l-z) (l-z2) 


since  0<z<x  or  y<  1. 

Assume  t(n)l  for  n=N,  then  the  first  difference  must  be  negative, 


x"-zW  xW+l-zN+l  xMri.vll.zll*U  +  xvN+l+  xzN1 
l-zN  '  l-zN+1  =  <l-zN)(l-zN+l) 


(l-zN)(l-zN+l) 


=>  A  >  o. 


For  n=N  +  1,  the  first  difference  is 


N+l  KM  1  N+2  N+2  N+lri  N+l  N+2  ^  N+\  N+2, 

x  -z  x  -z  x  II -y  -z  -x  +  xy  +  xz  I 


(l-zN+l)(l-zN+2) 


(l-zN+l)(l-zN+2) 


We  hav* 
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B-A  =  yN(l-y)  +  zN+'(l-z)  +  xyN+'(y-l)  +  xzN(z-l) 
=  0-y)y(M  +  (i-z)  (zN+,-xzN) 


=  (l-Z)yN(1-y)(i-xN+1)  >  0, 


N+l_  N+l  N+2  N+2 

soA>0=>B>0=>  - —  >  - — -r — ,  and  by  induction  f  (n) 

l-zN  '  l-zN  2 


is  strictly  decreasing. 


Writing  f(n) 


I-(f)n 


1-z 


-n 


and  exponentiating,  we  see  f(n) 


is 


convex  if  exp  (1-y  n)  exp  (z  n-1)  =  exp  (z  n-y  n)  =  g(n)  is  convex. 
Considering  n  as  a  continuous  variable,  n  >  1,  we  can  calculate 


*~2  9 (n )  =  [0  z)2(z"n  +  z~2n)  +  (1  y)2(y"2n-y’n)-2l  (yz)(yz)n>xp(z‘n-y”,t) 
dn 

>  0 

since  each  term  is  positive.  Thus,  g(n)  is  convex  if  n  can  assume 
aM  values,  equal  to  or  greater  than.  I,  which  implies  g(n)  is  convex 
for  n  a  positive  integer  by  using  embedding  arguments,  QED. 

As  a  result  of  this  theorem,  the  optimal  value  of  n  is  the  largest 
feasible  value,  subject  to  the  operating  constraint.  If  one  takes  the  atti¬ 
tude  that  operating  constraints  are  imposed  because  of  a  subjective  value  of 
the  holding  cost  h,  one  can  compute  an  operationally  valid  value  for  h  by 
calculating  what  value  it  must  assume  so  that  min  imizing  £(n  ;9 )  satisfies 
the  operating  constraints. 
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4.6  Cone  1  us  i  ons 


In  most  applications,  inter-arrival  and  service  times  will  not  be  so 
long  that  it  is  necessary  to  discount  costs  incurred  at  the  end  of  the 
interval,  and  the  undiscounted  policy  presented  in  Chapter  III  is  the  one 
that  should  be  used.  The  discounted  model  can  be  used  in  investment  problems 
where  one  Is  interested  in  the  expected  present  worth  of  the  future  costs 


of  a  proposed  system;  given  the  parameters  of  a  proposed  system 

(X,  p,  V2g,  R|,  etc.),  the  policy  for  using  the  system  will  be  n0^  from 

Section  3 . 3 »  and  equation  (10)  is  used  to  calculate  the  expected  present 


cost  of  operation. 


45. 


Chapter  V 

THE  FINITE  HOR  IZON  MODEL 

Where  there  is  a  finite  horizon,  T,  several  new  difficulties  arise. 

First,  the  optimal  policy  is  generally  non-stati onary ,  so  finding  the  best 
stationary  policy  is  not  sufficient;  secondly,  the  random  variables  or  and 
B  are  unbounded  so  a  transition  may  not  be  completed;  lastly,  it  may  be 
optimal  to  turn  the  server  off  while  there  are  customers  in  the  system,  so 
the  optimal  policy  for  the  imbedded  problem  may  not  be  optimal  for  the  full 
problem.  The  last  assertion  is  illustrated  by  an  example;  Suppose  a  Is 
constant  at  thirty  minutes;  twenty  minutes  remain  until  the  end  of  the  horizon; 
and  there  is  a  service  completion  with  k  >  0  customers  are  in  queue.  Since 
no  more  services  can  be  completed,  the  best  thing  to  do  is  turn  the  server 
off  and  save  r^-rj  Per  minute  for  the  remaining  ten  minutes.  These  con¬ 
siderations  make  it  very  difficult  to  calculate  optimal  policies,  but  for 
large  values  of  T  good  policies  can  be  found  by  using  asymptotic  results. 

5.1  The  Recursion  Formula  for  Optimal  Policies 

The  boundary  conditions  that  will  be  imposed  at  the  end  of  the  horizon 

are; 

(1)  A  charge  of  Q.[$ /customer]  for  all  customers  left  in  the  system 
[service  uncompleted  or  not  started], 

(2)  If  the  server  is  running,  it  must  be  turned  off. 

Define  the  state  of  the  system  as  (i,j),  where  j  is  the  number  of 
customers  present  and  i=i  indicates  the  server  is  dormant  and  i=2  means  It's 
running.  In  each  state,  the  possible  actions  are  *1=1,  turn  (or  leave)  the 
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server  off,  and  CL  =2 ,  turn  (or  leave)  the  server  on.  Let  y.j(c\.,t)  be 
the  expected  cost  of  a  transition  (which  may  not  be  completed)  leaving  state 
(i,j),  using  act  ct  ,  and  with  t(0  <  t  <  T)  time  units  remaining;  v.j(t)  is 
the  expected  to  :al  cost  starting  with  state  (i,j)  with  a  remaining  horizon,  t, 
and  following  an  optimal  policy. 

Letting  Y  be  the  number  of  customers  that  arrive  during  a  service 
interval,  and  %  be  the  number  of  customers  in  the  system  after  a  service 
completion,  for  0  <  t  <  T  we  have  the  recursion  relationships: 

0a)  y  1  j  ( 1  *t)  =  J  [  (jh  +  rj)o  +  Vj  j+1(t-cr)]Xe“XcydQ' 

+  C  0h  +  r ,)  t  +  jQ]e"Xt, 

Ob)  y2j0.t)  =  r2  +  y  i  j  ( 1  *  t) . 

(1c)  y,j(2,t)  =  R,  +  J  [  (jh  +  Y  h  +  r1  +  r2)o  +  v?  (t-a)ldB(a) 

+  C  (jh  +  j  +  r}  +  r2)t  +  R2lBC (t) , 


(Id) 


y2j(2.0  =  y,j (2»0  -  R,, 


t  k 

and  P(Y=k)  =  J  e"Xa  dB(a),  %  =  j-l  +  6(j)  +  Y. 


From  the  principle  of  optimality,  the  optimal  policy  must  satisfy  the 
usual  recursion  relationship  of  dynamic  programming  [10]: 


(2) 


V.j(t) 


Min  r.y;|(l,t),  y  .(2,0],  i-l ,  2 ;  0  <  t  <  T. 

1,2  ,J  IJ 
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Which,  in  principle,  can  be  built  up  for  successive  values  of  t,  since  the 


y.j  only  depend  on  pr i or  values  of  v.^(t). 


5.2  Conclusions 

There  is  no  general  way  to  obtain  solutions  to  equation  (2);  discrete 
approximations  of  the  continuous  variable  t  can  be  made,  and  digital  com¬ 
puting  methods  will  obtain  sufficiently  accurate  approximations  of  the 
integrals  so  the  resulting  policy  will  be  optimal. 

In  example  3,  Section  3.4,  we  showed  that  the  infinite  horizon  problem 
may  not  have  a  unique  optimal  policy.  This  means  that  as  T  -»  »,  the  optimal 
finite  horizon  policy  may  not  approach  the  optimal  infinite  horizon  policy 
as  a  limit.  That  is,  for  some  large  values  of  T  one  of  the  infinite  horizon 
policies  may  be  best,  while  for  s  me  other  values  of  T  another  policy  may 
be  best. 

Since  the  transitions  from  state  to  state  are  governed  by  an  ergodic 
(ireducible  and  positive  recurrent)  Markov  chain,  for  large  values  of  t 


(3)  vj  j  (0  «  C  (n) t  +  k.j 

where  C(n)  is  the  cost  rate  for  an  infinite  horizon  (Chapter  III)  and  k  is 
a  bias  term  that's  independent  of  t  but  dependent  on  the  policy  and  state 
(reference  10).  This  implies  that  an  optimal  policy  for  the  infinite  horizon 
problem  will  be  nearly  optimal  for  large  finite  horizons. 

This  conjecture  may  be  tested  by  seeing  r  equation  (3)  and  the  optimal 
pol icy  from  Chapter  III  satisfy  equation  (2). 


Chapter  VI 
A  tv/o-channel  MODEL 


Many  production  facilities  have  spare  machines  that  are  activated  when 
the  workload  reaches  a  critical  level;  these  machines  are  run  in  a  parallel 
with  normally  used  machines  until  the  workload  level  is  sufficiently  reduced. 
The  problem  of  specifying  the  workload  levels  that  trigger  activation  and 
deactivation  of  the  spare  machines  will  be  called  the  spare  machine  problem. 
Because  simple  probabilistic  results  can  only  be  obtained  when  the  service 
times  in  each  channel  have  an  exponential  distribution,  this  particular 
service  time  distribution  will  be  assumed. 

6.1  Assumptions 

The  assumptions  of  the  two-channel  spare  machine  problem  are: 

(a)  The  arr  iva  1  stream  -  Customers  arrive  in  a  Poisson  process  at  rate 
X,  and  form  a  single  queue. 

(b)  The  service  mechanism  -  There  are  two  servers,  machine  one  and 
machine  two.  The  service  times  in  each  machine  are  independent, 
exponential  random  variables  at  rates  and  p^  respectively,  with 
0  <  p.  j ,  ^2  <  ®  and  ^  <  1*  j  +  ^2  ’  Machi  ne  one  i  s  a  Iways  runn  i  ng  and 
machine  two  may  be  turned  on  and  off  arbitrarily.  If  machine  two 

is  turned  off  while  processing  a  customer,  that  customer  rejoins  the 
queue,  or  possibly,  directly  into  machine  one;  otherwise,  customers 
are  served  in  their  order  of  arrival  by  any  available  running 
machine . 

(c)  The  cost  structure  -  The  running  cost  rates  are  r  2  ^  t  $  /t  i  rne  "1  and 
r22[$/timel  for  machines  one  and  two,  respectively.  A  fixed  cost 
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of  Rj[$]  <s  charged  when  machine  two  is  tuned  on,  and  the  fixed 
cost  of  shut-down  is  A  holding  cost  of  h[$ /customer -hr .1 

is  charged  during  the  lifetime  of  each  customer.  The  costs  are 
non-negative  and  h  >  0  avoids  triviality;  future  costs  are  not 
di scoun  ted . 

(d)  The  dec  i s  ion  problem  -  When  should  the  spare  machine  be  turned  on 
and  off  to  minimize  the  cost  rate  over  an  infinite  horizon? 

6.2  Stationary  Optimal  Policies 

We  will  analyze  this  problem  by  imbedding  it  in  a  dynamic  programming 
problem  at  arrival  and  departure  epochs.  This  causes  no  loss  of  generality 
with  respect  to  making  decisions  at  arbitrary  time  instants  because  the 
assumptions  on  the  arrival  and  service  distributions  imply  that  the  time  to 
the  next  event  (arrival  epoch  or  service  completion)  has  an  exponential 
distribution.  Since  the  cost  rate  is  always  charged,  it  will  not  effect 
the  desirability  of  various  policies,  it  is,  however,  part  of  the  cost 
rate. 

Define  the  state  space  to  be  S  =  (0,  O',  1,  1',  ...,  k,  k', 
where  k  represents  the  number  of  customers  in  the  system;  an  unprimed  k  means 
the  spare  machine  is  dormant,  and  a  primed  k  indicates  the  spare  machine 
is  running.  The  action  space  is  A=(l,2),  where  action  Ot.e  is  nturn  (or 
keep)  the  spare  machine  off"  and  action  two  is  "turn  (or  keep)  the  spare 
machine  on". 

Using  the  methods  of  Chapter  II  one  can  prove  there  is  a  stationary 
optimal  policy  that  is  independent  of  the  initial  state  of  the  system.  To 
derive  the  form  of  this  policy  we  first  prove  two  lemmas. 
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Lemma 


Proof : 


0) 


(2) 


Lenma 


1_:  When  a  stationary  optima]  policy  is  used,  if  the  spare  machine  is 
turned  (or  left  on)  when  n  or  more  customers  are  present,  it  will 
not  be  turned  off  when  there  are  more  than  n  customers  in  the 
system,  n  >  2. 

Assume  a  stationary  policy  that  turns  the  spare  machine  on  in  state 
n  and  turns  it  off  when  n+k  (k  >  0)  customers  are  present,  and 
assume  this  policy  gives  a  cost  rate  g  =  C  (tt)  which  is  minimal. 
Obviously,  this  cannot  be  true  when  k=0,  so  we  only  have  to  consider 
k  >  1.  First  we  observe  that  as  the  horizon  approaches  infinity 
the  number  of  transitions  out  of  states  n  and  (n+k)'  approach 
infinity  with  probability  one,  so  that  if  an  improvement  can  be 
made  on  the  expected  cost  of  a  transition  leaving  state  (n+k)1,  the 
cost  rate  can  be  improved.  Using  act  2  in  state  n  implies 


r22+nh 


nh 


+  g 


r22+nh 


nh 


X4VIj4^2  -  X+Vij  ’ 


and  using  act  1  in  state  (n+k)1  implies 


nh 


r22+nh 


X+p .  —  X+p  ,+p 


+  kh 


p. 


1  ^2 


(X4p,  |+V-2)  (X+V 


which  contradicts  equation  (1)  for  any  k  >  1. 

Therefore,  using  act  2  in  state  (n+k)'  will  lower  the  cost, 
rate,  and  the  lemma  is  proved.  QED . 


.  When  a  stationary  optimal  policy  is  used,  if  the  spare  machine  is 
turned  (or  left)  on  when  n  customers  (n  >  2)  are  present,  it  will 
not  be  turned  on  when  two  or  more  customers  are  in  the  system. 
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Proof : 


(3) 


(4) 


Theorem 


Proof : 


As 

hibited 


As  a  consequence  of  lemma  1,  we  only  need  to  consider  turning  the 
spare  machine  off  when  there  are  m,  2  <  m  <  n,  customers  in  the 
system.  Assume  it's  optimal  to  use  act  1  in  state  (n-k)', 

1  <  k  <  n-2,  then 

(n-k)h  _  (n-k)h  <  r22 

X+iij  “  X+p- 14^  ' 

Combining  equations  (1)  and  (3)  we  obtain 

kh  [-(X-hij4^  "  ° 

wich  can't  be  satisfied  for  k  >  1 ,  so  act  one  cannot  be  optimal  in 
state  m'  =  (n-k)',  QED . 

J_:  A  stationary  optimal  policy  either  (a)  keeps  the  spare  machine 
on  at  all  times,  (b)  never  turns  the  spare  machine  on,  or  (c) 
has  the  form;  Turn  the  spare  machine  on  when  n  customers  are 
in  the  system,  and  off  when  m  <  1  customers  are  in  the  system. 

Lemmas  1  and  2  imply  that  m  <  1  when  n  >  2.  When  service  interruptions 
are  prohibited,  turning  the  spare  machine  on  when  n=l  cannot  be 
optimal  because  the  running  cost  rate  r^  ' s  incurred  without  re¬ 
ducing  the  expected  holding  costs;  the  only  remaining  policies  are 
those  where  n=0.  Since  turning  the  spare  machine  on  when  n=0,  off 
when  m'  =  l,  and  on  when  n=2  is  clearly  not  optimal,  all  possible 
stationary  policies  have  the  required  fcrm,  QED. 

a  consequence  of  this  theorem,  when  service  interruptions  are  pro- 
there  are  only  four  forms  the  optimal  stationary  policy  can  take, 


viz; 
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TTj  -  Never  turn  the  spare  machine  on. 

tTj  -  Leave  the  spare  machine  on  a  1 1  the  time. 

nj  -  Turn  the  spare  machine  on  when  n  >  2  customers  are  present,  and 
turn  it  off  when  the  system  becomes  empty. 

tt^  -  Turn  the  spare  machine  on  when  n  >  2  customers  are  present,  and 
turn  it  off  when  one  customer  is  left  in  the  system  and  he's 
being  served  by  machine  one. 

When  interrupting  service  and  switching  the  customer  to  the  other 
server  is  permitted,  three  more  policies,  call  swi tchinq  pol icies ,  may  be 
optima  1 .  They  are: 

TTj.  -  Turn  the  spare  machine  on  when  n  >  2  customers  are  present,  and 
turn  it  off  when  it  is  serving  the  only  customer  in  the  system, 

[  i  .e  . ,  restart  him  in  machine  one!. 

TTg  -  Turn  the  spare  machine  on  when  n  >  2;  if  machine  one  is  serv,,, 

the  only  customer  in  the  system  and  machine  two  is  running  switch 
this  customer  to  machine  two.  Turn  the  spare  machine  off  when 
the  system  is  empty. 

tt^  -  Turn  the  spare  machine  on  when  r.=  1  and  switch  the  customer  in 

machine  one  to  machine  two.  Turn  the  spare  machine  off  when  the 
system  is  empty. 


Policy  tt -j  is  a  switching  policy  because  we  assume  that  a  customer  who 
arrives  when  machine  one  is  idle  and  machine  two  is  dormant,  will  enter 


machine  one. 
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6.3  Calculation  of  the  Cost-Rates  when  Switching  is  Prohibited 

When  policy  TT|  is  used,  the  system  behaves  like  an  M/M/1  queue  with 
service  rate  pj,  and  the  cost  rate,  obtained  from  equation  (II 1-4),  is 

(5)  Cfrj)  =  r2]  +  ^  >  X. 

Since  this  policy  yields  an  infinite  cost  rate  when  p^  <  X,  TTj  cannot 
be  optimal  unless  p^  <  X. 

When  policy  tt2  is  used,  the  system  performs  like  an  I1/M/2  queue  with 
service  rate  p^  when  one  customer  is  in  the  system.  Using  well  known 
formulae  (see  reference  4,  Section  2.4),  the  expected  number  of  customer; 
present  in  this  type  of  queue  is  given  by 

L  =  0-p)(i+p,-p,)’  p  ’  pi  *  ^7 

the  cost  t ate  is 

P  jh 

=  r2l  +  r22  +  (1-p)  (l+p  ,-p)  ’ 

The  remaining  two  cost  rates  will  be  calculated  using  the  policy 
evaluation  routine  of  Markov-rcnewa  1  programming  [reference  10].  When 
either  of  the  policies  or  tt^  are  followed,  a  Markov -renewal  process  with 
additive  costs  during  each  transition  epoches  of  the  Queueing  process  so 
that  the  cost  rates  of  both  processes  ere  equal.  Furthermore,  theorem  l 
implies  this  can  be  done  in  a  manner  such  that  the  underlying  Markov  chain 
is  ergodic  (irrducible  and  positive  recurrent)  and  the  expected  time  for 
each  transition  is  finite.  For  an  infinite-time  process  operating  under  a 


I 


stationary  policy,  the  system  of  equations 


(7) 


N 

E 


Vi  +Vi  =yi  +  Pij 


.v , 


1=1 , 


•  •  • » 


N 
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where  v^  is  the  relative  value  of  state  i,  is  the  cost  rate  [previously 
called  C(tt.)3,  v.  is  the  expected  length  of  a  transition  leaving  state  i, 
y.  is  the  expected  one-step  cost  of  a  transition  from  state  i,  and  p.^  are 
the  conditional  transition  probabilities  of  the  underlyir.j  Markov  chain,  can 
be  solved  for  g  and  v.  uniquely. 

p.?1icy.jt.3 

When  policy  rr^  is  used,  the  imbedding  points  are  arrival  and  service 
epochs  when  only  machine  one  is  running,  and  turning  machine  two  on  is 
represented  as  a  transition  from  state  n  to  state  0.  The  corresponding 
Markov  chain  and  conditional  probabilities  are 

M-l 

(8)  Por1»  pn(f1’  pij  =  {  X  .  .  .}  I=2»  •••'  n"1» 

W,  J=,+' 

and  one  easily  finds  that 


(9) 


i=l 


n-1 


A  transition  from  state  n  to  state  zero  can  be  thought  of  as  a  transi¬ 
tion  from  n  to  one  followed  by  a  transition  from  one  to  zero.  The  first 
part,  (n  -»  1),  behaves  like  a  busy  period  of  an  ^ /G / 1  queue  with  service 


4. 
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rate  from  the  results  of  Chapter  I  1 1  we  have  that  the  expected 
length  of  this  interval  E  (n  -*  1),  and  the  expected  number  of  customers 
present  during  this  time  L(n  -  1),  are  given  by 


(10) 


E(n 


1) 


n  - 1 

|.»  j+p,2+x 


,  L(n 


l) 


n 

^1+Pl2“x 


The  second  part  of  the  transition,  (1  -•  0),  behaves  like  the  busy 
period  of  an  M/M/2  queue  where  the  service  rate  when  one  customer  Is  in  the 


1 


h 


system  is  p,.  with  probability  — - — ,  and  is  p,.  with  probability  — Let 
1  ^1^2  2  lii+M>2 

Ej  and  E^  be  the  expected  lengths  of  bu  y  periods  when  ther  service  rates 
are  and  auring  the  first  time  thire  is  only  one  customer  in  service. 
Conditioning  on  the  first  transition  leaving  state  1 1 ,  we  find  that  Ej  and 
E2  satisfy 


(n)  E<  =7i^T  +  ^.rfE(2-  '>  +^e2  +^ei]' 

(12)  E2  =T^  +  i^E(2"  0 +^E2 +^Eil: 

substituting  equation  (12)  into  equation  (11)  we  obtain 

2 

Gl.+p-o)  [(^2)  +  M>2  i^o) 

(13)  e  - — - - - - — — - r- 

(X+pL  | )  (X^t^j.  |  )  Cl^  ^  (m*  1  '^V'2  ^  ^  ~  ^(^2^  * 


,  X(pt1+n2)[  (X+p,2)  (M.^2)  +  p.2(lAl+^2+X^  E  ^2 
(X+VJ. |+\ji2)Cp. ^  (m<  |+M'2)  (x+M*2)  ”  x  (p.2) 23 


E2  is  given  by  a  similar  equation  with  the  sut  cripts  interchanged.  Thus; 
the  expected  time  to  go  from  state  one  to  state  zero  is 
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(14) 


E(1  -  0)  = 


^2  h 

^1^2  E]  +  ^1^2  Ez* 


Letting  Lj  equal  the  time -average  of  the  number  of  customers  in  the 
system  when  |Aj  is  the  service  rate  the  first  time  only  one  customer  is  in 
service,  we  find  that  Lj  and  L2  can  be  calculated  by  solving 

.  h  .  .  .  ^2 


(15)  Lj  = 


■ ..  4.  n  c ,  4.  c ,  1 

L (p, j+^2"^-)^  Uj+P'2  ^  ^  ^  ' 


(X+p-j  )Ej 


,  j  =  l,  2 


simultaneously,  and  the  average  number  of  customers  in  the  system  during  a 
transition  from  state  one  to  state  zero  is 


(16) 


L(1  -  0) 


V")  M-i 

—  L  E.  +  L.E. 

M,^+y*2  1  1  ^  1^2  2  2 

E(l  -  0)  ’ 


From  equations  (10),  (14)  ,  and  (16)  we  obtain 


07) 


Vn=i^x  +  E<'-°), 

yn  =  R,  +  R2  +  <r2!  +  r22)vn  +  h[j^“A  +  2  +  L(l  -  °>]v 


The  special  form  of  the  p.j  given  in  equations  (8)  allows  us  to  write 
the  first  n-1  equations  of  the  system  (6)  as  the  linear  second  order 
difference  equation 


X-ty, 


<l8>  vi+2  - -tvhi  +  rvi  = 


g  "r2i‘ih 


,  i  =0 . n-2 , 


with  boundary  conditions 
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(19) 


g 


The  solution  to  equation  (18)  is 


2p, 


(20)  v.  =  d1+d2(>Y1) 


rr2i~ 
L  n,-; 


g  h(3X-p,1) 


2  (n,-X)‘ 


■]' 


hi 


.2 


2(nj-XT 


1=2, 


where  the  coefficients  dj  and  d2  are  obtained  from  the  boundary  conditions 

09). 

Since  we  can  find  v  in  terms  of  q  from  equation  (20),  and  v  =y  -  v 
from  equation  (7),  the  value  of  g  =  C(rr)  can  be  obtained  as  a  function  of  n 
and  the  parameters  of  the  model.  One  can  then  find  the  optimal  n  for  this 
policy. 

Policy  TTf. 

The  values  of  v.  represent  the  relative  value  of  being  in  state  i  when 
machine  two  is  dormant;  for  policies  rr^  and  tt^  we  are  interested  in  the 
relative  value  of  being  in  state  i  when  machine  two  is  running,  denoted 
w. ,  and  particulary  in  w^ .  Observe  that  vt.  =  R2  +  Vj  . 

Let  w |  be  the  relative  value  of  state  1  when  machine  two  is  running  and 
busy,  and  w'j1  be  the  relative  value  when  machine  two  is  running  but  idle. 
Under  policy  tt^,  the  v.  satisfy  equation  (18)  and  the  Wj  are  given  by 


(21a) 


*0  =  R2 


(21b) 


wi  +  x-t,j2 


r„+r,,+h  |i,  , 

+  r— =-  w_  +  rr —  w 


x-hi. 


X-^2 


0  ■  "2 


w'l  ■  R2  +  v. 


(21c) 
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„  r ,+2h  p. 

(21d)  w  +  —  —8 - =  —— — — - +  — 1 — —  ( — 1 _ wii  +  ..  .-2..  wi^ 

2  X+p.^2  X+vtj+p-2  X+p,|+y,2  1  ^ 


+ - - - w 

X-fy.+p,,  3 


(2lcj)  w  ,  9  _  r22+r21  +  ih  ,  ^1^2  w  x  X  w 

'  i  X+p,^+y,2  X+^1^2  1 "  1  X+p,j-hi-2  ‘  +  ^ 


i =3 ,  4 ,  ...  n - 1 . 


Equation  (2 Id )  is  a  linear  second  order  difference  equation  whose 
solution  is 


u  n  in  ^1\i,  rr21+r22“  9  (3X ^  ^  .  hi 

(22)  w.  =  q1-K,2(“T~)  +  1.  u^l-x  +  77~T2J  1  +  TU: 


^1^2  “x  2(p,1-nx2-x): 


^  1+VJ'2"X^ 


,=4»  5  ,  •  •  n  , 

and  the  constants  and  q2  are  obtained  by  solving  for  w2  and  w^,  in  terms 
of  W  j  using  equations  (21a-d),  and  then  using  the  relationship  w^  =  +  R2 

to  ca leu  late  w j . 

Under  policy  tt^,  transitions  leaving  state  n‘  enter  state  1*  in  the 
imbedded  Markov-renewa 1  process;  using  equation  (8)  we  obtain 


fn-l)  n-l  r.  .  nu1  ,  “l 


(23)  "n  +  =  n|^j-xLr2l+r22+V)-ni2-X  +  2)h]  +  w”  +  wr 


Using  the  values  of  obtained  f rom  equat i ons  (22)  and  (23),  an 
equation  for  the  cost  rate  g  can  be  obtained. 

In  summary,  the  cost  rate  and  optimal  policy  of  the  form  rr^  is  found  by: 

(1)  Solving  for  the  v.(i=0,  ...,  n)  from  equati ons  (19)  and  (20), 

(2)  Sol  ving  for  thew.(i=0,  ...,  n)  from  equat  i  ons  (21)  and  (22). 
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(3)  Equating  the  values  of  wn  obtained  from  equations  (22)  and  (23), 
*  solve  for  the  cost  rate  in  terms  of  the  decision  variable  n. 

(4)  Find  the  value  of  n  (n  >  2)  than  minimizes  the  cost  rate. 

6.4  Calculation  of  the  Cost-fates  for  the  Switching  Policies 

Policy  TTj-  ii<  evaluated  with  the  same  equations  as  policy  and  the 
additional  relationship  wj  =  +  v^ ,  which  implies  wj  =  w'j'  =  Wj .  This 

simplifies  equation  (21)  to 

(24a)  wQ  =  R2 

(24b)  W1  =  R2  +  V1 

q  r22+r21+'h  ^1^2  \ 

(2hc)  w  +  2  =  Zml  +  1  —  ^  yj  +  -  -  yj 

v  X+vi  j+M*2  ^+*xl+^2 

i ”2 1  •  •  •  |  n-1 . 

0 

The  solution  to  equation  (24c)  is  given  by  equation  (22),  and  the 
constants  qj  and  q^  are  evaluated  from  the  boundary  conditions  (24a)  and 
(24b) : 

Equations  (23)  simplifies  to 

*25)  wn  +  =  Cr22+r21  +  ^,-tv,2“X  +  ^  +  W1* 

and  an  expression  for  g  can  be  obtained  using  the  values  of  wn  given  by 
equations  (24c)  and  (25). 

Policy  tt£  is  solved  in  an  analagous  manner;  the  boundary  condition 


60. 


(24b)  is  replaced  by 


(24d) 


“l  "  V1 


h+r21+r22‘J  +  X 


X+p,, 


X+y,,  2 ' 


where  is  given  by  equation  (20). 


the 


When  policy  tt ^  is  used,  the  system  behaves  like  an  H/M/2  queue  where 
service  rate  is  when  one  customer  is  in  the  system.  For  this  type 


of  queue,  the  probability  that  the  system  is  empty  p^,  and  the  expected 
number  of  customers  present  L,  are  given  by  [reference  4,  Section  2.4], 


(26) 


P  -  -1-P-  L  -  y  . .■? 

0  l+o2+p’  (l-p)  (l+p2-p) 


where  p0  =  —  and  p  =  — \ — .  Using  the  relationship 

i-  p.j  ^1*^2 


p  _  EK1 _ 1 _ 1 

o  _  eqo  xFfx)"  "  Y+\e(y) 


where  E (X)  and  E  (y)  are  the  expected  lengths  of  a  busy  cycle  and  a  busy 
period  respectively,  one  obtains 


(27) 


1+Po-P  1 


From  equations  (26)  and  (27),  we  obtain  the  cost  rate  of  policy  tt^, 


hp, 


r  22^2  + 


(28)  C(n7)  -  r2|  +  (l-p)  (lip  +p)  +  l+p,-p 


Jt**  . 
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6.5  Numerical  Examples 

Example  1;  X=l,  ^=^2=1,  r22=5,  r2)=2»  h=t°>  R|=*2=0 

When  switching  is  prohibited,  a  policy  of  the  form  which  turns  the 
space  machine  on  when  two  customers  are  present  is  optimal,  and  the  cost 
rate  is  14  yy  [$/hr.].  If  switching  were  allowed,  switching  to  the  only 
customer  from  machine  two  to  machine  one,  policy  rr,- ,  would  lower  the  cost 
rate  to  13  j  [  $  /hr .]  . 

Example  2;  X=l,  p,j  =  l ,  p-2=2 ,  r22=5,  r2 ^  =2 ,  h=10,  RpO,  R2=5 

When  switching  is  prohibited,  a  policy  of  the  form  tt^  is  optimal; 
turning  the  spare  machine  on  when  two  customers  are  present  and  turning  it 
off  when  the  system  becomes  empty  gives  a  cost  rate  of  19  [$/hr,].  When 
switching  is  allowed,  a  policy  of  the  form  rr^  is  optimal,  and  the  cost  rate 
is  reduced  to  12.6  [$/hr.]. 

6.6  Conclusions 

Although  closed  form  expressions  are  not  obtained  for  the  cost  rates 
of  the  seven  policies  considered  in  this  chapter,  we  are  able  to  give 
qualitative  relationships  among  the  cost  parameters  that  will  indicate  the 
optimal  policy. 

Policy  tt,  will  be  used  when  h  is  small,  X  <  p,j,  and  the  operating  costs 
of  the  spare  machine  are  large;  while  rr2  will  be  more  advantageous  when  the 
fixed  costs  of  the  spare  machine  and  the  holding  costs  are  large,  and  the 
running  cost  is  small.  Policy  TTj  will  be  used  to  hedge  against  high  waiting 
lines  when  all  the  costs  are  of  the  same  relative  importance.  Policies  rr^ 
and  TTj.  protect  against  high  holding  costs  but  incur  the  fixed  costs 
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frequently;  rr^  will  preferred  to  tt^  when  the  running  cost  of  the  spare 
machine  high. 

Policies  tt^  and  tt^  tend  to  lower  the  cost  rate  when  the  spare  machine 
is  faster  than  the  regular  machine  and  its  running  cost  rate  is  small. 
Policy  tt^  wi  1 1  be  preferred  to  policy  tt^  when  the  fixed  costs  of  the  spare 
machine  are  large. 

One  should  not  be  rvslead  about  the  difficulty  of  finding  the  cost 
rates  for  policies  rr^,  tt^,  tTj.  ,  and  rr^;  numerical  solutions  will  be  easy  to 
obtain  since  all  the  equations  are  linear  in  the  cost  rate.  The  major 
difficulty  wi 1 1  be  to  calculate  the  optimal  value  of  n  since  calculus 
methods  lead  to  implicit  equations  for  n ... 


SUMMARY 


The  aim  of  this  thesis  is  to  describe  the  economic  behavior  of  a 
controllable  system  wi th  a  linear  cost  structure,  and  to  find  cost-minimiz¬ 
ing  policies  for  turning  the  server  on  and  off.  The  costs  considered  are: 
a  server  start-up  cost,  a  server  shut-down  cost,  a  cost  per  unit  time  when 
the  server  is  turned  off,  a  cost  per  unit  time  when  the  server  is  turned  on, 
and  a  holding  cost  of  waiting  customers.  Single-channel  queues  with  Poisson 
arrivals  and  arbitrary  service  time  distributions  are  emphasized;  Chapter 
Six  is  devoted  to  a  two-channel  system  with  Poisson  arrivals  and  exponentially- 
distributed  service  times. 

In  Chapter  Two  the  decision  process  for  a  single-channel  queue  operating 
for  an  infinite  horizon  is  formulated  as  a  dynamic  program.  We  shew  that 
when  future  costs  are  discounted,  there  exists  a  stationary  optimal  policy 
to  minimize  the  expected  total  cost;  we  use  this  result  to  prove  that  when 
discounting  is  not  used,  there  is  a  stationary  optimal  policy  that  minimizes 
the  cost  rate.  We  then  prove  that  for  both  models,  the  stationary  optimal 
policy  has  the  form:  Turn  the  server  on  when  n  customers  are  present,  and 
turn  it  off  when  the  system  is  empty. 

Chapter  Three  deals  with  infinite-horizon,  undiscounted  models.  For 
models  where  the  arrival  stream  is  a  Poisson  process,  two  methods  for 
deriving  the  cost  rate  as  a  function  of  the  decision  variable  n  are  pre¬ 
sented,  and  the  optimal  value  of  n  and  the  minimum  cost  rate  are  obtained. 

We  consider  decreasing  the  ,'ected  service  time  with  an  increase  in  the 
running  cost  of  the  server;  an  expression  showing  when  the  server  should  he 
speeded  up,  or  slowed  down,  is  given.  When  the  inter-arrival  time  distri¬ 
bution  is  generalized  to  the  class  of  IFR  distributions,  we  obtain  narrow 
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bounds  for  the  optimal  expected  cost  rate. 

An  equation  for  the  expected  discounted  cost  over  an  infinite  horizon 
is  derived  in  Chapter  Four;  we  prove  that  for  small  interest  rates,  the 
undiscounted  policy  will  be  a  good  approximation  to  the  optimal  policy. 

When  the  horizon  is  finite,  the  optimal  policy  is  generally  non¬ 
stationary.  A  recursion  relationship  to  find  the  optimal  policy  when  the 
horizon  is  small  is  presented,  and  the  optimal  undiscounted  policy  for  the 
infinite  horizon  is  shewn  to  be  a  good  approximation  to  the  optimal  po  licy 
for  large  horizons. 

The  last  chapter  is  devoted  to  a  two-channel  service  system,  where 
each  channel  is  restricted  to  have  an  exponentially  distributed  service 
time  (possibly  with  different  rates),  and  the  arrivals  form  a  Poisson 
process.  One  server  is  always  turned  on;  the  other,  the  spare  machine, 
can  be  turned  on  and  off  at  arbitrary  times.  Using  a  dynamic  programming 
formulation  of  ;he  decision  process,  we  shew  that  the  stationary  optimal 
policy  for  undiscounted  costs  has  the  form:  Turn  the  spare  machine  cn  when 
n  customers  are  present,  and  turn  it  off  when  m  customers  are  in  the  system, 
with  m<  1.  We  derive  equations  for  finding  the  optimal  value  of  m  and  n 
when  service  interruptions  are  prohibited;  then  we  consider  queue  disciplines 
where  customers  nay  be  switched,  without  delay  or  cost,  from  one  server  to 
the  other. 

In  addition  to  being  applicable  to  policy  problems  for  existing  systems, 
these  models  should  be  useful  when  comparing  proposed  investments  in  service 
system  because  they  relate  the  parameters  of  the  server  to  the  cost  of  the 
facility.  The  most  promising  areas  for  future  research  appear  to  be: 
systems  where  arriving  customers  may  not  enter  the  queue  if  the  waiting 
line  is  too  large,  systems  where  customers  leave  the  waiting  line  if  they 
have  not  been  served  after  a  given  wait  in  queue,  and  processes  w<th 
different  types  of  customers. 
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